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EDITORIAL NOTE 


To surpass success is difficult, but through teamwork and co-operative effort it can be done, as in the case of the Second Congress 
of the International Federation of Automatic Control. 

The first LF.A.C. Congress was held in Moscow in June-July 1960 and attracted some 1500 participants, and 286 papers which 
were published in four volumes of some 2000 pages only 15 months later; this First Congress set a high standard for subsequent 
ones to match. 

Yet as judged by its participants, the Second LF.A.C. Congress held at Basle in August-September 1963 did indeed match these 
standards. Although the number of papers was only 159, around 1500 participants accompanied by 200 ladies came from 32 different 
countries to attend these presentations. 

Editing these papers into their present form also presented difficulties if the high standard attained with the Proceedings of the 
First Congress was to be maintained. A lot of these difficulties are, of course, common to all international congresses, but these 
become aggravated when several languages are used for the original manuscripts of the papers. In the Second Congress, 36 papers 
were originally submitted in Russian for publication in English, while nine papers were written and published in French. To make 
the operation truly international, an English Publisher was chosen working in conjunction with a German Printer. 

Other difficulties that plague Editors of Congress Proceedings are delays in receiving the manuscripts, rapid translation require¬ 
ments, and the lack of translators sufficiently expert in the intricacies and detailed knowledge of the field involved. All these problems 
were present when preparing the preprints of the papers for the Basle Congress. As a result, translations in the preprints were often 
not of the standard that one would have desired, since the inflexible publication date for the preprints prevented execution of revisions 
and changes that should have been made. 

Even with all the goodwill and co-operation that the Printers and Publishers gave to the project, the 600 mile separation and 
language difference took their toll. Many typographical errors resulted in the preprints as there was no time for authors to receive 
and return proofs before the Congress. In spite of conscientious work by both Publisher and Printer after the Congress to check 
the 1*5 million words in the Proceedings, to improve the translations and to correct the typographical errors, it could be that some 
traces of these pre-Congress difficulties are still visible. If so, the Editors express their sincerest apologies for any errors that may 
still be found in these volumes. 

Editing of the discussion remarks was greatly eased by the outstanding organization of the Congress by the Swiss Federation of 
Automatic Control, Professor Ed. Gerecke, Third I.F.A.C. President (1961-1963) and by the Congress Secretary Dr. A. von Schult- 
hess and the I.F.A.C. Honorary Secretary Dr. G. Ruppel. The main burden in preparing the edited discussion remarks was shoulder¬ 
ed by Dipl.-Ing. E. Ruosch of Zurich. Leading an efficient team of scientific secretaries from the Eidgenossische Technische Hoch- 
schule of Zurich, the Technische Hochschule of Darmstadt and the University of Cambridge, Dipl.-Ing.' Ruosch succeeded in 
supplying, often less than 48 hours after the presentation of each session, typewritten documents in English of all the discussion 
remarks on each paper in the session. 

In the tedious and painstaking work of editing these volumes, the work of the Co-editors, Otto Schafer of Aachen and Derek 
Barlow of London, has been invaluable, especially that of Mr. Barlow. Located in London and hence closest to the Publisher, 
Mr. Barlow bore the brunt of the detailed editing work, in particular the editing of the discussions on the applications and com¬ 
ponents papers. Editing of the final form of the Theory papers was done through the co-operation of the U.S.S.R. National Com¬ 
mittee on Automatic Control. 

Thanks are due also to the Publisher, particularly for the large amount of detailed sub-editing carried out by his co-operative 
staff, and also the Printer in overcoming the problems of printing scientific works in languages that were not of his own country. 
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EDITORIAL NOTE 


It is hoped that, despite these problems and their effect on the Proceedings, the latter will become a long-living valued contri¬ 
bution in the development of the science of Automatic Control. The excellence of the Congress and the quality of the contributions 
certainly deserve this recognition. 

It has been especially gratifying to the Editor and his colleagues to participate in what has been for them, as well as for all who 
attended the Congress, one of the most rewarding of human experiences—a fruitful international co-operative effort. 


Boulogne sur Seine, France 
June 1964 


Victor Broida 
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The Second International Congress of LF.A.C. 

in Basle 1963 

EDUARD GERECKE, Third President of LF.A.C. 


This, the second I.F.A.C. Congress, is being held in Basle from 
27th August to 4th September. 1476 participants and 200 ladies 
are present from the following countries: Argentine (1), Austria 
(11), Belgium (37), Bulgaria (8), Canada (5), China (10), Congo 
(3), Czechoslovakia (11), Denmark (10), Finland (25), France 
(173), Germany (212), Hungary (30), India (1), Israel (4), Italy 
(47), Japan (19), Mexico (1), Netherlands (83), Norway (22), 
Poland (32), Portugal (3), Roumania (10), Spain (6), Sweden (44), 
Switzerland (237), Turkey (7), United Kingdom (160), United 
States of America (154), Union of Socialist Soviet Republics (80), 
Yugoslavia (30). 

As Automatic Control today covers a very large field, the 
Executive Council of LF.A.C. selected the following limited 
fields for the Congress. 

1. Theory 

2. Applications 

3. Components 

As an innovation, 11 Survey Papers dealing with the actual 
state of automatic control in most fields of theory, applications 
and components will be read. All Congress delegates can attend 
these lectures, and will, I am sure, certainly appreciate the op¬ 
portunity of getting a competent survey from outstanding special¬ 
ists of today’s position and of future developments in the differ¬ 
ent fields of Automatic Control. 

Two hundred and sixty Discussion Papers were submitted to 


LF.A.C. in September 1962, and the LF.A.C. Selection Com¬ 
mittee accepted 159 of these, namely 82 on Theory, 57 on Ap¬ 
plications, and 20 on Components. Fifty half-day sessions are 
planned for the discussion of these papers, 25 of them on 
Theory, 19 on Applications and 6 on Components. Undoubtedly 
the LF.A.C. Basle Meeting will contribute to the promulgation 
of the new and more advanced chapters of Automatic Control. 

The organization of such a large meeting requires the co¬ 
operation of a large number of individuals. The I.F.A.C. officials 
and many members of the Technical Committee did a great deal 
of preparatory work, all on a voluntary basis, and their help is 
very much appreciated. Our sincere thanks also go to the Honor¬ 
ary Secretary, Dr. G. Ruppel, Dtisseldorf, for his indefatigable 
secretarial work and to the Honorary Editor, Professor V. Broida, 
who prepares the Discussion Papers and the Proceedings with 
the aid of the Co-editors, D. H. Barlow, London, and Professor 
O. Schafer, Aachen. I should like also to thank the Scientific 
Secretary, E. Ruosch, Zurich, for his most valuable assistance. 

The expenses of the Congress, including the Preprints and the 
Survey Papers, amount to U.S. $ 97,000 for which the Council 
members of the Swiss Society of Automatic Control are personal¬ 
ly responsible. As third President of LF.A.C. I should like to 
thank most heartily the Swiss Federation of Automatic Control 
and all those who contribute financially, especially the Authori¬ 
ties of the Canton of Basle Town, the Swiss Industries Fair and 
the Swiss Industry. 
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How I.F.A.C. was Founded 

H. CHESTNUT, First President of I.F.A.C. 


It is appropriate as we start this Second Congress of I.F.A.C. on 
Automatic Control that we look back at some earlier inter¬ 
national meetings on automatic control and see what we may 
learn from the past that will give us some ideas that will be use¬ 
ful for the future. Within the past twenty-five years the science 
and art of automatic control has grown rapidly and interest has 
developed in such subjects as regulation, automation, and auto¬ 
matic control on the part of engineers, politicians, and laymen. 
We who are developing and using the techniques of automatic 
control are, in a way, charged with the responsibility of making 
automatic control most effective. It is important that the place 
of international activities, including meetings and congresses, be 
considered as a part of the process of helping us to accomplish 
our job of making automatic control better. 

Although the first General Assembly of I.F.A.C. was held in 
Paris in September 1957, a number of meetings that were inter¬ 
national in character if not in name had been held prior to that 
time. In 1951 at Cranfield, England, was held an outstanding 
meeting on ‘Automatic and Manual Control’ sponsored by the 
Department of Scientific and Industrial Research, the Proceed¬ 
ings of which were edited by Professor Arnold Tustin. In 1953 
in New York under A.S.M.E. sponsorship was held a ‘Frequency 
Response Symposium’ that was international in its list of authors; 
its Proceedings were edited by Dr. Rufus Oldenburger. 

By 1956 the heightened interest in automatic control caused 
a number of meetings to be held in Europe on or related to this 
subject. Outstanding among these was ‘Regelungstechnik’ in 
Heidelberg, Germany, in September 1956. 

An earnest effort was made to obtain broad international 
participation on the part of authors from many countries serv¬ 
ing to bring together many leading men in automatic control. 
This had the effect of increasing their interests in the possibilities 
of more regular and more internationally organized meetings on 
automatic control theory and practice that would be more broad¬ 
ly publicized and attended. 

It was appreciated that similar progress in automatic control 
was being made in many parts of the world, in many cases with 
each group of people largely oblivious of the works of the others 
or without even knowing who these other people were. Further¬ 
more, it was apparent that a great amount of time and effort 
would be required to push forward our understanding of auto¬ 
matic control and our ability to use it. Hence the help and benefit 
to be obtained by the sharing of ideas and information on an 
international basis would be valuable in making for more rapid 
progress. 

Fortunately, a group of dedicated men, including Dr. Victor 
Broida, Chairman; Dr. Otto Grebe, Professor A. M. Letov, 


Professor P. J. Nowacki, Dr. Rufus Oldenburger, Dr. G. Ruppel, 
and Mr. D. B. Welbourn, were willing to devote their energies 
during the period September 1956 to August 1957 to helping in 
bringing about the formation of the International Federation of 
Automatic Control at the first General Assembly of I.F.A.C. in 
Paris, September 1957. 

At this initial meeting a Constitution was adopted built 
around the principle of national rather than personal represen¬ 
tation, and particular emphasis was placed on the holding of 
periodic international congresses. Another important objective 
was ‘to promote the science of automatic control by the inter¬ 
change and circulation of information on automatic control 
activities in co-operation with national and other international 
organizations’. Plans were laid for the first I.F.A.C. Congress 
to be held in Moscow, U.S.S.R., in 1960 about which you will 
hear shortly from Professor A. M. Letov who was President of 
I.F.A.C. at that time. The first I.F.A.C. Congress was a very 
successful one from which was obtained a fine, four volume set 
of Proceedings edited by John F. Coales. 

The International Federation of Automatic Control is indeed 
fortunate that the Swiss Federation of Automatic Control was 
willing to serve as host for this second I.F.A.C. Congress here 
at Basle. The preparation and labour involved by members of 
the Swiss Federation of Automatic Control for this Congress 
are outstanding. On behalf of all those present here, I want to 
thank Professor Gerecke in his dual role as President of I.F.A.C. 
and as President of the Swiss Federation of Automatic Control 
for his help and for the efforts and skill of the members of the 
two groups which he represents and directs. 

As first President of I.F.A.C., I am very pleased with the way 
that I.F.A.C. has developed and with the interest which its 
Congresses have stimulated. The growth of I.F.A.C. symposia 
on special subjects has been of particular interest to me. I look 
forward with hope for additional symposia on suitable topics of 
concern to experts in the field of automatic control. 

We all have reason to be pleased with the fine progress that 
has been made in our automatic control accomplishments to 
date. Lest we become complacent, however, I should like to point 
out two other fields for serious attention by control people. 
These are: 

(1) The need for ‘optimizing the process of making auto¬ 
matic control, i.e. bridging the gap between theory and practice. 

(2) The need for working with qualified people in the social, 
economic, and political fields to help make the net effect of 
automatic control and automation a cause for hope rather than 
a reason for fear. 
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THE FIRST I.F.A.C. CONGRESS 


For the past 5 years and more, one of the most popular topics 
of automatic control investigation has been the subject of opti¬ 
mum control. With all this emphasis on optimum control, it 
would appear possible to apply some of these general principles 
to permit us to perform the process of designing and building 
automatic controls more quickly, more cheaply, more reliably, 
or more favourably in some other sense which allows the desired 
balance of a number of these objectives. Not only should concern 
be given to determining how to make a system which will be 
optimum, but also the process of design itself should be one 
which can be readily applied by the large number of designers 
who will be applying these ideas to the making of better controls. 
Attention must be given from the theory point of view to includ¬ 
ing the practical application of these new control concepts. 
Efforts must be made from both sides to bridge the gap between 
theory and application. 

The second problem which deserves more of our attention 
is that of automation, the popular term by which much of 
automatic control is known. Throughout the centuries, Man has 
hoped to find a way of obtaining goods and services with a 
minimum of effort to himself. With the advent of more and more 
automation, we are approaching the condition where a signifi¬ 


cant proportion of the necessary production and services can be 
achieved with a minimum amount of direct human effort. 

To realize the opportunity that automation can afford will 
require more than just the technical attention that we, working 
in automatic control, can apply. The changes in production and 
services brought about by automation will also involve changes 
in the way people live and earn their livelihood. I believe it will 
be more effective if engineers and scientists skilled in automatic 
control systems work with people skilled in dealing with social, 
economic and political problems to help bring about the needed 
changes in a smooth and socially acceptable fashion. 

Unfortunately, the human time constant is one of the longest 
we have to deal with. Although the problems associated with the 
introduction of more widespread automation are great. The op¬ 
portunities for a better world at peace make the challenge of 
using automation for the betterment of man one that is certainly 
worth working for. 

From what I have seen of the preprints of the papers for this 
Congress, I am looking forward to an interesting week of dis¬ 
cussions here. I am hopeful that the creativity and vision that 
have characterized I.F.A.C. from its beginning will continue to 
grow and flourish as we move ahead with this second I.F.A.C. 
Congress and on to the future. 


The First I.F.A.C. Congress 

A. M. LETOV, Second President of I.F.A.C. 


Three years have passed since the Moscow Congress—the first 
I.F.A.C. Congress, which brought together 2,000 specialists from 
28 countries; and at which some 300 papers dealing with the 
solution of major scientific and engineering problems of autom¬ 
ation, were read and discussed. 

Although reports of the Congress were published in news¬ 
papers and magazines in many countries, and the Congress 
Proceedings were published in Russian and English, I now re¬ 
member the Congress not merely because it brought great satis¬ 
faction to those who took part, or because the aims of I.F.A.C. — 
so well expressed in the speech of our President, Professor 
Gerecke— were realized so widely for the first time in Moscow. 

I recall the Congress for another reason; because, as the poet 
said, ‘pleasant recollections are the fount of good inspiration’, 
from which we draw new strength to develop the future activity 
of our Federation. The conditions for developing this activity 
are becoming more and more favourable. 

We have now assembled at our second Congress. It has been 
organized by the Swiss National Federation, headed by our 
respected President Professor Gerecke. But the activity of the 
Federation is not characterized by this alone. The activity of 
I.F.A.C. is characterized, in particular, by the willingness of 
many other countries to organize subsequent I.F.A.C. Con¬ 
gresses. Such Congresses will undoubtedly be held. 

Although it may be in 90 years time, I still hope fervently 


that I can live to the noteworthy day when the I.F.A.C. Congress 
will have gone round all the countries in our Federation and 
returned once again to Russia—perhaps to Moscow—involv¬ 
ing not 2,000 but 20,000 participants. I also look forward to the 
day when the linguistic difficulties of communication will have 
been overcome by the creation of miniature ‘radio-computers’ 
which will translate into one’s own language the speeches of 
representatives of all countries on the globe; and to the day when 
science and technology will make it both pleasant and inspiring 
to look forward to what is to come a century ahead. 

You will say that it is a very remote dream. As yet it is a 
dream, I grant you. Let me just say this. First, you are all people 
who do creative work—dreamers—and all the plans you imple¬ 
ment so wonderfully begun with a dream. 

Secondly, let me, by speaking of my dreams, give those with 
a sense of humour a chance to say, what else can a man do who, 
after 6 years of helping to run the Federation, and for the mo¬ 
ment still its Past President, but relinquish the authority in three 
days when he retires. 

Thank you, Mr. President, for the opportunity given me here 
to dream aloud; to my audience I say thank you for your atten¬ 
tion. 

With all my heart I say, ‘I look forward to meeting soon at 
a new Moscow Congress, dear colleagues’. 
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The Information Revolution and Its Impact on Automatic Control 

I. L. AUERBACH, President of the IFIP 


The invention of the electronic digital computer in 1946 marked 
the beginning of the information revolution. The ensuing seven¬ 
teen years have seen a development very much like the industrial 
revolution that followed the advent of the steam engine in 1765. 
In both instances, the advancement of civilization had created a 
growing need for new ways of accomplishing vital tasks. The 
major technological breakthrough not only filled the need, and 
opened new avenues in many fields, but led to the discovery of 
new fields, where development would have been out of the 
question without the new tool. Thus it was that the electronic 
computer brought about a revolution in information processing, 
rather than just advanced normal development in the field. 

The essential feature of the industrial revolution was man’s 
amplification of his brawn by the use of engines. In the infor¬ 
mation revolution the emphasis has shifted to the amplification 
of brain through computers and information processing systems. 
Already there are as many different kinds of computers as there 
are kinds of engines. The applications of digital data processing 
are limited only by the ingenuity of scientists and technologists 
in their respective technical fields. 

There is a basic and fundamental difference between brawn 
and brain—a difference that is exaggerated when these faculties 
are extended and amplified by mechanical engines or electronic 
processors. A muscle or an engine consumes energy to accom¬ 
plish work. The fuel can never be recovered or re-used. In 
contrast, information handling is non-destructive. Information is 
used and applied without being consumed. It can be used over 
and over again, and many of the applications augment the 
original supply, but none can diminish it. 

The words in a book, for example, remain intact, and can 
be read by any number of people without loss of information 
content. The responses the words elicit in different minds may 
even go beyond the original content. Each new edition of a great 
book may have more footnotes than the one before it—words 
growing on words as they strive to capture ideas. Similarly, a 
computer manipulates information in a scientific computation, 
and generates new information from it. The information is never 
consumed by being processed; it can be retrieved and used 
repeatedly in many different ways. 

This difference perhaps explains why the information re¬ 
volution has been faster and more widespread in its implications 
than the industrial revolution. Electronic data processing is so 
broad a subject that only an infinitesimal fraction of it can be 
explored in a short time. I would like to take a brief look at the 
field where the engine and the computer work together; the 
impact of the information revolution on automatic control. In 
addition, at the request of your President, I will review the back¬ 
ground of the International Federation for Information Process¬ 
ing (IFIP) and discuss the areas of cooperation between that 
body and the International Federation for Automatic Control 
(IF AC). 

Figure 1 shows the basic structure of an automatic control 
loop. In it the status of a physical system is detected through 


sensors or transducers and transmitted to a device labeled ‘Com¬ 
puter’. This device identifies any differences between the indicated 
status of the system and its desired status, and activates controls 
to modify the physical system. The changes in status resulting 
from this modification are in turn sensed and transmitted to the 
computer, and so the operation of the loop goes on. The com¬ 
puter not only can guide the system through a series of steps, 
but can modify any future step on the basis of the results of 
previous steps. 
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Figure L The automatic control loop 

This simplified diagram illustrates how the computer per¬ 
forms the functions of a human operator. With its fantastic speed 
of reaction and calculation, its resistance to fatigue or distraction, 
and the great variety of inputs and outputs it can utilize, the 
computer can replace a human operator in many physical 
systems. More important, it can assume new tasks too taxing for 
human beings and thus make new physical systems feasible. The 
launching of a satellite, for example, simply could not be handled 
by anything short of an automatic computer system. 

Still man’s place in automatic control is a vital one. The loop 
shown in Figure 1 is usually part of a more complex loop as 
shown in Figure 2. The ‘desired status’ mentioned in the discus¬ 
sion of Figure 1 is determined by man. He observes the physical 
system, and applies his intelligence, his ability to judge situations 
rather than respond to them deterministically, and his set of 
values. He.establishes the criteria and the end goals to be achiev¬ 
ed by the system, and communicates his conclusions to the 
computer via programming. This is a critical input to the com¬ 
puter. It cannot be absent, as might be inferred from Figure i, 
although it may precede in time the actual operation of the 
system. 

This powerful combination of man, computer, and physical 
system multiplies the resources of all three. Automatic control 
combines the benefits of the industrial revolution and the in¬ 
formation revolution, giving man an extension of his brain and 
his brawn, both applied to the same task. The digital com¬ 
puter makes this system quite distinct from the man-engine 
combination alone. It handles great quantities of data at tre¬ 
mendous speeds, extending the realm of operations that can 
be performed. Programming techniques also combine decision 
making with mathematical calculations, and this permits dis- 
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Figure 2. The man loop 


continuous or discrete control to be intermixed with the various 
kinds of continuous or smooth control that are afforded by 
analogue devices. 

It can be seen in Figure 2 that man’s portion of the loop, 
proceeding from the physical system to the computer, parallels 
the direct sensing of system status by the computer. Both con¬ 
nections are necessary, as each exploits the individual character¬ 
istics of its medium: man or machine. Some of the outstanding 
developments in recent computer history are directed toward 
man ’s turning over as many as possible of his own functions 
to the computer. Much of the programming process is being 
made automatic, and can be classified as computer reaction to 
a system status. 

The computer may be multi-programmed, that is, may time- 
share or interleave a number of independent or interrelated 
programmes run at the same time. It is now feasible to have one 
computer run a programme that controls the physical system 
based on the results of an independent programme being run 
at the same time that derives economic and business criteria. 
For example, if the inventory and order control for a business 
were maintained by the same computer, then the output of the 
physical system would be based on the current requirements and 
permit very rapid response times. In addition, multi-computer 
systems are becoming more commonplace, and with appropriate 
communication links further broaden the variety of tasks that 
can be handled by computers in the automatic control loop. 

The programming of the computer in the automatic control 
loop is far more complex than most people anticipated. The 
programme is actually interposed between the computer and the 
process or physical system it is to control. It is a necessary link 
in the computer controlled system. In this area, there is need for 
more work in automatic programming techniques and compilers 
specifically oriented toward process control. Multi-programme 
and multi-computer systems will add to the complexity and impose 
the need for far more sophisticated interrupt features and the 
techniques for their application. The skill required to formulate 
the computer programme is major and is far more difficult in 
complex interconnected computer systems. It should be clear that 
greater training for people capable of doing this work is essential. 

It must be recognized that more and more is being done by 
computers that it was thought only human beings could do. 
Pattern recognition is one such extension of computer capabili¬ 
ties. There exist today devices for recognizing printed symbols in 
a number of different type founts without human assistance. 
This enables the introduction of text from a printed page into 
a computer, where formerly translation into a digital form such 
as holes in paper was necessary. Similarly, it is possible for 


computers to read maps and match them with the areas they 

represent. . . 

Rather than belittle the activities of man’s brain, these imita¬ 
tions of thought by computers show man’s superior intelligence 
more clearly. The programming of artificial perception calls for 
not only the same perception on man’s part, but a transcendent 
awareness of how that perception works. By reflecting on the 
way his own mind operates, man has been able to reduce many 
of its functions to the sort of simple instructions a computer 
can follow. 

This has not been an easy process. In language translation, 
for example, great difficulties were encountered. Linguists were 
not immediately aware of how they translate. But linguists and 
computer engineers are consulting and coming to grips with the 
problem, and teaching both participants more about language 
than was ever known before. Completely new methods of lin¬ 
guistic analysis are opening up, which will be as useful to future 
language study itself as they are to computer applications. 

Artificial learning is another field that has come into being 
as a result of mechanizing thought processes. The computer in 
the three-way loop can modify its own programme according to 
occurrences in the physical system. A man can programme a 
computer to play a game and then be beaten by the computer. 
Investigations in this area are shedding new light on the problems 
of educational psychology. 

The area in which computer technology can be expected to 
make the most significant contribution to automatic control is 
in the optimization of continuous processes. This optimization 
will be achieved in two ways. The computer’s computational 
capabilities will be utilized to make more effective use of data 
on the internal variables of the process. The computer also has 
the completely unique ability to help man to optimize a process 
on the basis of variables external, but related, to the process. 

Process optimization on the basis of internal variables is 
coming about through the development of computer program¬ 
mes that continually modify process parameters in the light of 
empirical experience. These are really automatic learning pro¬ 
grammes in which the computer uses a model of the process to 
predict the response of a change in an operating variable. The 
computer then makes the change in the variable, observes the 
response (through sensory instrumentation), and modifies pre¬ 
dictive methodology in the light of its empirical experience.. 

Process optimization on the basis of external variables is a 
concept that depends entirely upon the computer s unique 
capabilities. Going well beyond the concept of integrating the 
various loops in an individual process, work in this area is 
directed at integrating several individual but related processes, 


XXXIX 






I. L. AUERBACH 


plus such business variables as product orders, delivery, and 
inventory. 

The benefits that can result from these two approaches to 
process optimization are obvious. However, the problems of 
realizing these benefits are extremely formidable. Success hinges 
upon how well we can integrate man, machine and the process 
into an effective closed loop. To do this requires an inter¬ 
disciplinary effort on the part of the control, operating, and 
computer engineers. 

Much work must be done on control theory and on formaliz¬ 
ing the operating methods that are still largely intuitive. Before 
we can programme computers to effectively perform the process- 
control functions, we must deepen our understanding of the 
basics of continuous processes, and we must be able to establish 
explicit criteria for defining an ‘optimum’ condition. We must 
also learn how to translate the ‘feel’ human operators have for 
the process into computer programmes. 

The effective application of the digital computer to process 
control is not only an analytical problem; it is also a software 
problem of major proportions. After the analyses, it is the 
computer systems engineers who must accept the final responsi¬ 
bility for developing and applying the proper software and pro¬ 
gramming skills, such as multi-programming and multi-computer 
programming, that will effectively relate the computer, the 
processes and the man to each other. 

In a limited way, computers are reciprocating the effort to 
make them behave like men. They seem to be showing men how 
to behave like computers. The quantities of information handled 
in modem data processing systems have necessitated new ap¬ 
proaches to the problems of indexing, storing, and retrieving 
information. The newly developed methods can be applied to 
situations not involving electronic computation at all. Even if 
all the electronic computers in the world ceased to exist, the 
effects of the information revolution would still be felt, and 
would exercise their influence on the future of civilization. 

The potentialities of this revolution can best be realized with 
maximum communication within the field. This interchange of 
ideas should not be inhibited by national boundaries. It calls for 
international cooperation and international organizations such 
as IFIP and IF AC. Now I will turn to a discussion of the inter¬ 
national Federation for Information Processing, the first inter¬ 
national organization dedicated to all facets of the information 
processing sciences. As defined by IFIP, these information pro¬ 
cessing sciences include theory, mathematics, equipment, and 
application—all applied to the collection, transmission, com¬ 
putation, translation, storage, retrieval, reduction, and display 
of information. 

As stated in its statutes, the aims of IFIP are threefold: 

(a) Sponsor international conferences and symposia on in¬ 
formation processing, including mathematical and engineering 
aspects. 

(b) Establish international committees to undertake special 
tasks falling within the spheres of action of the member societies. 

(c) Advance the interests of member societies in international 
cooperation in the field of information processing. 

One of the goals of IFIP is to expose the people of the world 
(those who will be affected by information technology as well as 
those directly associated with it) to some idea of the progress 
that can be made through the intelligent use of the electronic 
digital computer. We hope to make an increasingly greater 


number of people aware of the information processing sciences 
and the benefits that can be derived from them. 

In achieving the above aims, IFIP fulfills the need for better 
worldwide communication and increased understanding among 
scientists of all nations of the role information processing can 
play in accelerating technical and scientific progress. It is hoped 
that better dissemination and centralized control of information 
about computer technology and application techniques will 
result in greater scientific advances, which will achieve the two 
purposes of benefiting mankind and advancing the state of the 
art. 

The first International Conference on Information Process¬ 
ing, sponsored by UNESCO, in June, 1959, provided a forum 
for the meeting of 1800 delegates from 37 countries. During the 
planning for this conference, it became apparent that future 
international meetings and other activities were essential to the 
development of the information sciences in many countries of 
the world. 

On June 18, 1959, recognizing the importance and success of 
the UNESCO conference, representatives of the computer socie¬ 
ties of 18 countries met in Paris to formulate the preliminary 
stmcture of IFIP. Statutes for the federation were drafted, and 
upon the agreement of 13 national technical societies (six more 
than the minimum required), IFIP came into official existence 
in January, 1960. 

IFIP Congress 62 

The IFIP Congress 62 was attended by more than 2800 
scientists from 41 nations, who were exposed to a comprehensive 
survey of the technical achievements and goals that have been 
made possible and practical by the digital computer. This ex¬ 
perience demonstrated to them the profound effect that the 
information revolution is having upon mankind. 

In addition to the technical sessions at the Congress, the 
computer scientists and users attending had an opportunity to 
view an exhibition of the progress being made in hardware 
development. The INTERDATA exhibition included exhibits 
of 48 companies from eight nations. This exhibition, with its 
emphasis upon hardware, complemented the material presented 
in the technical sessions on the software and application aspects 
of information processing. 

For the practical realization of its goals, IFIP has established 
three technical committees and one working group. The scope 
and accomplishments of these technical committees are sum¬ 
marized below. 

IFIP TC-1 Terminology 

The scope of this committee is the establishment of termi¬ 
nology of digital computers and data processing devices, equip¬ 
ments, media and systems. The objective is to promote the ex¬ 
change of information, leading to the compilation of a multi¬ 
lingual glossary for information processing systems and related 
subjects. 

The specific programme of work of this committee includes: 
select natural languages for which the terms shall be defined; 
collect documentation of pertinent glossaries; systematize a 
master list of terms and concepts requiring definition; create or 
adapt terms for missing concepts and assign these terms to one 
or more specific fields; choose, modify dr originate accepted 
definitions. 


XL 



THE INFORMATION REVOLUTION AND ITS IMPACT ON AUTOMATIC CONTROL 


To avoid duplication of effort, the IFIP Committee on 
Terminology was affiliated with a similar committee within the 
International Computation Centre of Rome to form TFIP/ICC 
TC-1 Terminology’. 

This committee has met seven times to date, and by the end 
of 1963 will have defined and assigned words in the English 
language to approximately 1000 concepts. Then specific language 
area groups will translate these concepts into the major national 
languages so that the particular words may be chosen in what¬ 
ever language descirbes each of the concepts most accurately. 
It is expected that a vocabulary of 1000 concepts will be published 
during 1964. 

It is significant that the multilingual glossary being developed 
by IFIP/ICC TC-1 Terminology has been officially requested by 
the International Standards Organization through subcommittee 
1 of its technical committee 97. ISO will use this glossary as the 
basic input in their effort to establish an international standard. 

IFIP TC-2 Programming Languages 

The scope of this committee is to promote the development, 
specification, and refinement of common programming languages 
with provisions for future revision, expansion and improvement. 
This specific programme of work includes: general questions on 
formal languages such as concepts, descriptions and classifica¬ 
tions; the study of specific programming languages; and the 
study and, where appropriate, coordination of the task of de¬ 
veloping new programming languages for which there apeears 
to be a need. 

Working Group 2.1 ALGOL 

During the past year, a series of meetings was held between 
the original ALGOL authors and other experts who clarified 
and amplified certain aspects of ALGOL '60 language, removing 
ambiguities that existed. Through IFIP TC-2 the working group 
submitted the ALGOL '60 document to the council of IFIP 
which approved the document and made ALGOL '60 official 
IFIP language. Once again the International Standards Organi¬ 
zation, through Subcommittee 5 of Technical Committee 97, 
has requested that certain specific additions to the ALGOL 


language be considered and then submitted to ISO for considera¬ 
tion as an international standard. 

IFIP TC-3 Education 

At the August 1962 meeting of the Council of IFIP it was 
decided to form a new committee on education. The objective 
of this committee ist to establish comprehensive training pro¬ 
grammes and suggested curricula for the education of technical 
people from all over the world who are in fields in which the 
computer can make a significant contribution. Another function 
of the committee will be to generate material to acquaint the lay 
public with the computer and its impact on the various aspects 
of society. This committee will, in fact, serve as a central clearing 
house on all educational material pertaining to the information 
processing sciences. In this capacity, it will assist in preparing 
or providing translations, lists of available material, and other 
necessary information services. 

The membership of each of the above IFIP committees is 
international, assuring each national group the opportunity to 
review and comment on all committee work before it reaches 
final, rigid form. This policy provides the committees with the 
additional advantage of having a consensus that includes the 
viewpoints of many diverse backgrounds. 

The activities of IFIP require the time, effort and interest of 
many people from many parts of the world, all of whom are 
devoting their services willingly and without material reward. 
They are to be commended on their accomplishments, for through 
their endeavours they are demonstrating that people from diverse 
national backgrounds can work together and communicate 
effectively to achieve a worthwhile international goal. 

There are many areas in which the work of IFIP and that of 
IFAC impinge upon each other. Cooperation can take place 
through the common membership of the same scientists in the 
national technical societies who are members of both our re¬ 
spective federations and by more direct cooperation of our 
technical committees. It is hoped that ways will be found in the 
coming years to increase the cooperation and coordination of 
the activities of these two international federations, and so to 
provide means for each of the federations to achieve its own 
goals more effectively and more quickly. 
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Statistical Methods in Automatic Control 

A Survey by V. S. PUGACHEV 


Introduction 

Every automatic system performs its function, subject to the 
permanent effect of various noises in its elements, as well as 
external interferences, natural or artificial, sometimes specially 
organized. Hence statistical methods are especially important 
for modern automatic control theory. 

Statistical methods afford the evaluation of automatic sys¬ 
tems accuracy, as well as the determining of optimal system 
characteristics, which provide maximum possible accuracy for a 
given statistical behaviour of signals and noises. Without statis¬ 
tical methods it is impossible at present to design complicated 
systems. For example, the development of modern theory of 
self-adaptive systems is possible only on the basis of statistical 
methods. Only statistical methods afford the determining of 
dynamic characteristics of plants and control systems from nor¬ 
mal operating records with real inputs (more precisely the de¬ 
signing of the best, in a sense of a certain criterion, models for 
given systems, because the dynamic characteristics of any system 
are subject to continuous random variations, and no absolutely 
precise measurements are available). 

Statistical methods are necessary not only for the design of 
control systems, but in many cases for organizing control pro¬ 
cesses, i.e. for the shaping of control signals in the systems 
elements. This is especially important for complicated systems 
containing many elements and communication channels trans¬ 
mitting information among the elements of a system. Control 
signals in such systems may be essentially corrupted by noises 
because of a great many independent noise sources and random 
variation of parameters of elements. Hence the problem of the 
best possible extraction of signals from noises arises. This prob¬ 
lem can be solved only by applying optimal statistical informa¬ 
tion processing. Thus the statistical methods are necessary in 
certain cases for the shaping of control signals in automatic 
systems. 

It is likely that such processes as learning, self-improvement 
and experience storage have a statistical nature. Thus statistical 
methods are also necessary for simulating processes of learning, 
self-improvement and experience storage in automatic systems. 

Statistical methods yield also the solution of problems of the 
automatic check of control processes, in particular in industry, 
and problems of automatic detection and determination of defects 
with the purpose of their timely reparation. 


It is therefore impossible to solve cardinal problems of auto¬ 
matic control without statistical methods, it is impossible to de¬ 
sign many complex automatic systems without using statistical 
algorithms. 

Yet the statistical methods are of a complex nature, and their 
application is generally bound with using high-speed computers. 
As a consequence, statistical methods are not yet so widely used 
in automatic systems design as more elementary deterministic 
methods. Statistical algorithms of information processing are 
often too complicated and cannot be used in automatic systems, 
owing to low speed and insufficient memory capacity of modern 
computers. As a consequence, statistical algorithms are seldom 
used in automatic systems for signal processing. 

In spite of the difficulties connected with using statistical 
methods in the theory and the practice of automatic systems 
design, the field of application of statistical methods is continu¬ 
ously widening. As a result, the statistical theory of automatic 
control systems is now one of the most important sections in 
modern automatic control theory. With the development of 
analogue and digital computers the use of statistical methods 
becomes more general. With the increasing speed of computers 
statistical methods will be used more in automatic systems for 
signal processing. 

Owing to a great variety of automatic control problems, 
which can be solved by statistical methods, the development of 
statistical methods in automatic control is proceeding in many 
directions. Thus it is scarcely possible to present a full survey 
of all directions of the development and the fields of application 
of statistical methods in automatic control in a brief report. 
Hence only the most important general statistical methods, 
according to the opinion of the author, will be considered here, 
many useful and necessary works in this field being omitted, 
for which the author hopes to be excused. 

The main statistical problems in automatic control theory, 
which are being developed at present, are the following: 

(1) The statistical analysis of systems with known dynamic 
characteristics. 

(2) The statistical synthesis of automatic systems, their ele¬ 
ments and optimal algorithms of control and information pro¬ 
cessing. 
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(3) The determining of dynamic characteristics (optimal 
models) of processes and plants. 

(4) The statistical theory and methods of simulating learning 
processes, self-improvement and experience storage processes. 


Statistical Analysis of Linear Systems 


The methods of the statistical theory of linear systems based 
on the correlation theory of random functions are the simplest 
statistical methods of automatic control systems analysis 1 . The 
system and its operator may be considered as deterministic, if 
all internal noises in its elements may be replaced by equivalent 
random inputs. To evaluate the accuracy of a linear system in 
such cases it is sufficient to know its matrix of weighting func¬ 
tions g (t 9 t), the expectation (the mean value) m x (t) and the 
correlation function K x ( t , t') of the vector of random inputs 
X (t). Then the expectation /% (/) and the correlation function 
K y (/, t') of the vector of outputs of the system Y ( t ) are deter¬ 
mined by 


My (0 = 


" t 

g(t,r) m x (r)dr 
J to 


( 1 ) 


K y (t, t') = 


g (t, t) K x (t, a) g (f a) dtdcr 


( 2 ) 


where ^ denotes a transposed matrix 1, Sect * 90 . 

The calculations of eqns (1) and (2) may be generally ac¬ 
complished with the aid of analogue computers after determin¬ 
ing the weighting functions matrix g (r, r). To determine weight¬ 
ing functions, the method of simulating adjoint systems may be 
used 1-3 . 

Eqns (1) and (2) may be applied to continuous linear systems 
as well as to sampled-data ones. The weighting functions matrix 
g ( t , r) is a linear combination of (5-functions of r in the latter 
case, the coefficients being functions of t in general. 

Another method for the evaluation of the accuracy of linear 
systems is based upon the expansion of random inputs in a 
series, whose members are uncorrelated elementary random 
functions (canonical expansion): 


X(t) = m x (t) + Y J V v x v (t) (3) 

V 


where V v are uncorrelated random variables (which are also 
statistically independent in the case of a normally distributed 
random function X), and x y (t) are certain definite functions, 
called coordinate functions. The canonical expansion of the form 
of eqn (3) may always be found for any random function (scalar 
or vector) in an infinite variety of ways, if its correlation function 
is given 1, Ch - 9 . In the case of the infinitesimal members of the 
expansion the sum in eqn (3) is replaced by the integral (integral 
canonical representation): 


X(t) = m x (t) + 


V{X)x{t,X) cU 


(4) 


where V (2) is a random function of a parameter 2 with the cor¬ 
relation function of the type of (5-function (the generalized white 
noise), and x (?, 2), considered as functions of t for various 
values of 2, are definite (non-random) coordinate functions. 

The problem of shaping a given random function from a 
physical white noise using analogue computers represents the so- 
called problem of finding a shaping filter. 


The generating of a given random function by passing a 
physical white noise through a shaping filter is thus a special 
way to find an integral canonical representation of this random 
function 2 . Vector random functions as well as scalar ones may 
be expressed by integral canonical representations, the coordi¬ 
nate functions x ( t , 2), as well as the coordinate functions x v (t) 
of a canonical expansion (3), being vector functions in the pre¬ 
vious case. 

Expressing the random function X by the expansion (3) or 
the integral (4) and using the superposition principle, the evalu¬ 
ation of the accuracy of a linear system is reduced to the deter¬ 
mination of its responses to the expectation of the input vector 
and to all coordinate functions x v (/) or x ( t , 2). The correlation 
function of the output vector is then given by 


K y (t, t , )=Y J D v y v (t)y v (t’) 

V 

or respectively by 

Ky(t, 0 = 


G (a) y ( t , A) y (f \ 1) cU 


(5) 

( 6 ) 


where D y are variances of random variables V v9 G (2) is the in¬ 
tensity of the white noise V (2) (i.e. the quantity by which the 
(5-function is multiplied to give the correlation function of this 
white noise), and y v ( t ), y (t , 2) are vector coordinate functions 
expressed as matrices-columns, representing the responses of the 
system to respective coordinate functions x v ( t ), x (t, 2), the bar 
denoting complex conjugates. 

To determine the functions m y ( t ), y y (/), y (/, 2) in the case 
where the weighting functions matrix of the system is known, 
the formula (1) may be used, as well as analogous formulas 


.Vv0)= 

*t 

g(t, i)x v (x)dT 

J to 
ft 

(7) 

y{t,X) = 

g(t, T)x(T,A)dr 

J to 

(8) 


In the special case of a sampled-data system the weighting 
functions matrix g (r, t) in eqns (7) and (8) represent a linear 
combination of the (5-functions of r whose coefficients are func¬ 
tions of t in the general case. 

To determine the functions m y ( t ), y v (t ), y ( t , 2) in the case 
where the weighting functions matrix of the system is unknown, 
analogue computers or the linear system itself may be used. 
Passing in turn through a given system or its simulator the 
functions m z (t ), x 2 (t) 9 ..., or the functions x (h 2) cor¬ 

responding to certain chosen values of 2, the respective functions 
m y (0> - v i ( 0 > ^2 (0, ..or y (t,X) are obtained. 

It often occurs in practical problems, especially in those con¬ 
nected with automatic control in industry, that linear systems 
and their inputs are stationary or may be assumed to be station¬ 
ary. The evaluation of systems accuracy is essentially simplified 
in this case, any stationary random function being representable 
by the integral canonical representation (4) with exponential 
coordinate functions (harmonic oscilations): 

x(t,X) = e nt (i = V- 1) (9) 


The intensity G (2) of the white noise V (2) coincides in this case 
with the spectral density s x (2) of the stationary random function 
Sect. 76 ^ similarly any stationary random vector function (i.e. 
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a vector random function with stationary and stationarily cor¬ 
related components) may be expressed by the integral canonical 
representation in the form of the sum of the integrals of the form 
of eqn (4) with the vector coordinate functions 

x(t,A) = a(X)e m (10) 

where a (2) is a vector function of the parameter ?h Sect - 78 . 

The coordinate functions y (t, A) are easily determined in this 
case, because harmonic oscillations pass through a stable station¬ 
ary linear system, being multiplied by the complex frequency 
response of the system. As a result the formula (6) shows that 
the output vector of the system is stationary, and its spectral and 
cross-spectral densities matrix is determined by the well-known 
formula _ 

s y (co) = 0 ( ico ) s x (co) $ ( ico ) (11) 

where 0 (1) is the matrix of the frequency responses of the sys¬ 
tem 1 ’ Sect - 95 . Formula (11) is evidently applicable only to a- 
symptotically stable systems in a steady state. To determine 
variances and covariances of the outputs of a stationary linear 
system in a transient state or of an unstable system the general 
formula (6) must be used, in which the functions y (t, A) are the 
transient responses of the system to the harmonic oscillations (9) 
or (10) 1 Sect - 90,92 . 

The theory of stationary random processes is thus insufficient 
in principle for basic automatic control problems, and only 
general theory of non-stationary random functions may serve 
as a mathematical basis of the statistical methods in automatic 
control theory. 

The methods of evaluating the accuracy of linear systems 
with known dynamic characteristics may be considered at present 
as completely developed. The above formulae yield the precise 
solution of the problem for any linear system. Although these 
formulae require tedious calculations, these calculations are 
easily accomplished with the aid of analogue or digital com¬ 
puters. 

Hence more convenient methods of calculations and more 
adequate algorithms now are the main subject of development 
in the theory of the statistical analysis of linear systems (see, for 
example, Nekolny and Benes 4 ). 

The knowledge of the first and second moments of system 
outputs is generally sufficient to characterize the accuracy of the 
system. In particular, the probability distribution of the outputs 
of a linear system is completely determined by these moments 
in the case of normally distributed inputs, because the distri¬ 
bution of outputs is also normal in this case. 

If the input distribution is not normal, the knowledge of the 
first and second moments is then insufficient to determine the 
output distribution, and the moments of higher orders may be 
calculated using formulae analogous to (2), the respective 
moments of inputs must of course be known in this case. 

It should be once more emphasized that the methods outlined 
above are applicable only to deterministic linear systems or to 
such systems whose internal noises and random variations of 
parameters can be replaced by equivalent random inputs. To 
determine the accuracy of linear systems with randomly changing 
dynamic characteristics, the statistical theory of systems reduc¬ 
ible to linear should be used (see section on Precise Statistical 
Theory, and ref. 1, Sect. 106, 107 and ref. 5). 


The Approximate Methods of Statistical Analysis of Non-linear 
Systems 

The statistical analysis of non-linear systems is an extremely 
difficult problem, and the precise theory of non-linear systems 
applicable to any non-linear systems does not exist at present. 

The relative simplicity of the methods of statistical analysis 
of linear systems is the natural reason for trials to extend these 
methods for the problems of approximate analysis of accuracy 
of non-linear systems. Hence various methods of linearization 
of operators of non-linear systems have arisen. 

The simplest form of linearization technique is based upon 
the expansion of all non-linear functions in the equations of a 
system (i.e. characteristics of all non-linear components) in 
Taylor series with respect to fluctuations, neglecting all members 
of the second and higher degrees. As a result, each non-linear 
function in the equations of the system cp (. X ) or cp (X lm ..., X n ) 
will be replaced by the approximate linear expression 


<p(X)x<p (m x ) + cp' (m x ) (X - m x ) 

or respectively 


cp(X u X„)x(p{m xi ,m x J 


It 


+ Z <Pxv 0*1’ ’ 

V — 1 


m x ,)(X v -m Xv ) 


( 12 ) 


(13) 


where m%, m Xl , ..., m Xn are expectations of the random functions 
X, Xj, ..., X n respectively, just as in the previous section. 

Using the formulae of the form of eqns (12) and (13), the 
equations of a system are linearized with respect to the fluctu¬ 
ations of the signals in various system components. This enables 
us to use the methods of the statistical theory of linear systems 
for approximate evaluation of the accuracy of non-linear systems. 
This gives non-linear equations for the expectations of signals, 
which coincide with the equations describing the behaviour of 
the system considered, as the expressions (12) and (13) are linear 
with respect to fluctuations only and are non-linear with respect 
to expectations 1 ’ Sect * 100 “ 102 . 

The accuracy of the results obtained by the linearization 
technique based upon eqns (12) and (13) may be estimated by 
the magnitudes of errors of the expressions (12) and (13) in the 
range of practically possible values of the signal fluctuations. 

Eqns (12) and (13) are valid only for continuous functions 
having continuous first derivatives with respect to all arguments 
in the domain of practically possible values of these arguments. 
Yet automatic systems often contain essentially non-linear com¬ 
ponents the characteristics of which are discontinuous (as, for 
instance, relays) or possess discontinuous derivatives (as, for in¬ 
stance, limiters). To linearize such characteristics the method of 
statistical linearization has been developed, proposed at first by 
Booton in U.S.A. 6 and by Kazakov in U.S.S.R. 7, and8,ai ‘ 3 ; 4 . 

The statistical linearization consists in replacing a non-linear 
component by a component linear with respect to fluctuations, 
conserving in a certain sense the magnitudes of output signal 
and output noise. 

The most general form of the statistical linearization tech¬ 
nique is based upon replacing all non-linear functions in the equa¬ 
tions of a system by the approximate linear ones with respect to 
fluctuations expressions of the form 9, and 8 Sect - 24,25 


cp(X(t))vcp 0 (t) + 


rt 

g(t,r) [X (t) — m x (t)] dt 

J t 0 


(14) 
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+ Z (t, r) [*, (0 - m Xv (0] dt (15) 

V = 1 J to 

The simplest form of the statistical linearization is Booton’s 
statistical linearization, which is obtained from (14) by putting 

(p 0 (t) = km x (t), g(t,x) = kd(t-z) (16) 

where k is determined so as to minimize the mean square error: 

MO(A)-/cA} 2 ] = min (17) 

M [. Z ] denoting the expectation of the random variable Z 6 . 

Kazakov’s statistical linearization in its first variant is ob¬ 
tained from (14) by putting 


<Po (0 = (0 > g t ) = k x 5(t- t) (18) 


for k x and l x , where D X} D x and K xx are, respectively, variances of 
the random functions X, X and the covariance of their values 
corresponding with the same instant of time. 

The conditional minimization of the mean square error con¬ 
serving the variance of the output yields the same eqns (24) with 
the additional factor X in their right members. This factor is 
determined by the condition (23) after solving eqns (24). 

The third possible way to determine k x and l x is to determine 
them from the condition of conserving the variances of the out¬ 
put and its derivative. 

All methods described above give for (p 0 the formula 

cp 0 (t) = M[cp(Xm (25) 

In the case of the general statistical linearization according 
to eqn (14) Kazakov obtained from the condition of conserving 
the expectation and the correlation function of the output the 
formula (25) for the function cp Q ( t ) and the equation 


and determining k 0 and k x so as to conserve the expectation 
and the variance of the output: 

M \^k 0 m x + k 1 (X — m x )-] = M [<p (X)] } 

D [k 0 m x +k l (X-m x ))=D [<p (A)] J 1 } 

where D [Z] denotes the variance of the random variable Z. 
Kazakov proposed also to determine k Q and k x , minimizing the 
mean square error: 

M [{<p (X) — k 0 m x — k x (X - m x )j 2 ] = min (20) 

Two conditions (19) and (20) giving a single value for k 0 and 
two different values for k l9 Kazakov recommends the use of the 
arithmetic mean of these two values of k x 7 . 

It should be emphasized that the previous statistical linear¬ 
ization of Booton and Kazakov based upon replacing a zero- 
memory non-linear component by a zero-memory linear one 
distorts the correlation function of the output and, as a conse¬ 
quence its spectrum in a stationary case. To conserve the spec¬ 
trum of the output Pupkov proposed replacing a zero-memory 
non-linear component by a memory stationary linear system in 
a steady state, that is, to put g (t, r) = w (t — r), t 0 = — oo in 
(14) and to determine w (f) (or corresponding transfer function) 
by equating the correlation functions of the left and right 
members of eqn (14) 10 . 

An interesting generalization of Kazakov’s statistical linear¬ 
ization is obtained by putting in (14) 

g(t,x) = k 1 d(t-z) + l 1 S' (t-t) (21) 

The condition (19) is insufficient to determine k x and l x in this 
case. They may be determined by minimizing the mean square 
error 

M iiv ( x ) -9o-ki(X- m x )~ (£ - mj} 2 ] =min (22) 

or by minimizing the mean square error under the additional 
condition of conserving the variance of the output: 

D j> 0 + fcj (X - m x ) + Ij. (Jf - mj] =D[q> (X)] (23) 

The condition (22) yields the equations 

D x k l+ K xi l^Ml(X- mx )cp(X)2 \ 

‘ K xi k 1 +D i l 1 =Ml(X-m x )cp(X)'] J ' 4J 


a) g (t, r)g(t',cr) dtder = K y (t, t") (26) 

J to J to 

for the weighting function g (t, t), where 

K y (t,t')=M[<p(X(t))<p(X(t'y)-J 

-M[<p(A'(0)]M[<p(Z(f'))] (27) 

is the correlation function of the output of the non-linear com¬ 
ponent 8, Sect - 24,25, and 9 . The minimization of the mean square 
error yields in this case the same formula (25) for cp 0 and the 
equation 

m t 

g(t,T) K x (t, a) dt = K yx (t, a) (f 0 <<r<f) (28) 

J to 

for the weighting function g ( t , r), 
where 

K yx (t, <j) = M [{A (<t) - m x (<t)} (p (A (f))] (29) 

is the cross-correlation function of the output and input of the 
non-linear component 8, Sect - 24, 25, and 9 . 

It should be noted, that in the case of a single-valued func¬ 
tion cp Booton’s and Kazakov’s first statistical linearization, 
minimizing the mean square error gives as a special case the 
harmonic linearization of a non-linear component, provided 
that the sinusoidal input is assumed as random with a definite 
amplitude and uniformly distributed over the period random 
phase. This is easily seen from the works of Pervozvansky 11 . 

More general statistical linearization according to formulas 
(14) and (21) minimizing the mean square error gives as a special 
case the harmonic linearization of a non-linear component in 
the case of a multi-valued characteristic (p (X) as well (hysteresis 
component, for instance). 

The above formulae show, that the statistical gains of a non¬ 
linear component k 9 k 0i k l9 l x as well as its statistical weighting 
function g (t, t) depend on the probability distribution of input, 
which cannot be known a priori , before the evaluation of the 
system accuracy. To determine these quantities Booton and 
Kazakov recommend assuming the normal distribution for in¬ 
puts of non-linear components. Then the coefficients k, k 0i k ly 
h w iH be functions of the unknown expectations and variances 
of inputs, and the application of the statistical linearization and 
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the methods of the statistical theory of linear systems yields a 
sufficient set of equations to determine all quantities that are 
unknown 1 ’ Sect * 103-105:8 » Secfc - 32 - 3 4-36; 12 s Sect. 11-2. 11-3 

The statistical linearization of Kazakov has been general¬ 
ized by Sommerville and Atherton for the case where the 
input of a non-linear component is the sum of independent 
random signals with different probability distributions 13 . Their 
statistical linearization in a generalized form is expressed by the 
formulae 

X(t)= t Xy(t) ( 30 ) 

v= 1 

n ft 

<P (X (0) «(po (0 + £ Sv (t, r) [X y (t) - m Xv (t)] dr (31) 

V - 1 J to 

Sommerville and Atherton have considered the case, which is 
obtained from (31) by putting 

<Po(t) = kom x (t),g v (t,x) = k v 5(t-x) (v=l, (32) 

The condition of conserving the variance of the output is 
evidently insufficient for the determination of the weighting func¬ 
tions g v (t, r) or, in Sommerville and Atherton’s case, of the 
coefficients k v . So the minimization of the mean square error is 
expedient for this purpose. Using this criterion, Sommerville 
and Atherton obtained the expressions for the coefficients k y in 
(32). It should be noted, that the coefficients k v may be determin¬ 
ed in this case also by the conditional minimizing of the mean 
square error conserving the variance of the output. 

An interesting generalization of Sommerville and Ather¬ 
ton’s statistical linearization is obtained from (33) by putting 

g v (t,x)=k v 8(t-x)+l v 6'(t-x) (v=l,...,«) (33) 

Then the statistical linearization gives as a special case the joint 
statistical and harmonic linearization, if the input of a non¬ 
linear component is the sum of a sinusoidal signal with a definite 
amplitude and uniformly distributed over the period random 
phase and a normally distributed noise. This joint statistical and 
harmonic linearization is generally used to determine the effect 
of noises upon the parameters of oscillations of non-linear sys¬ 
tems 12 ’ Sect ‘ 10 ‘ 7:14>Ch - 10: 8> Sect * 38: 1L Sect. 3*2, 3-3 

All the methods outlined above yield the same expression (25) 
for^o- 

For the weighting functions g y ( t , r) in (31) the minimizing 
of the mean square error yields the system of linear integral 
equations, which may be written using matrix notations in the 
form of eqn (28). 

The development of the statistical linearization method and 
its applications have been given in numerous papers by scientists 
of various countries. Among these the work of Japanese scientists 
Sawaragi, Sunahara and others should be especially noted 15-19 . 

Based upon various forms of linearization, approximate 
methods of the analysis of non-linear systems accuracy are the 
simplest and the most effective methods of the statistical theory 
of non-linear systems, owing to which they have been widely 
expanded. It is impossible to give even the enumeration of all 
contributions to this field in this brief report. 

The natural way to increase the accuracy of the statistical 
methods of non-linear systems analysis based upon linearization 
is to use the members of the second and higher degrees in the 
series expansions of non-linear functions. So the method of 


series expansions of outputs of non-linear systems with respect 
to random parameters 1 ’ Sect 108 and Dostupov’s method of 
equivalent disturbances 8 ’ Ch * 11 ’ and 20 have arisen. The latter 
is based upon simulating some typical realizations of random 
inputs to a system chosen in such a way, that some linear com¬ 
binations of various degrees of resulting system outputs with 
definite coefficients be equal to corresponding moments of 
system outputs. Owing to this Dostupov’s method is especially 
fit for the evaluation of non-linear systems accuracy with the aid 
of digital computers. 

Another method of non-linear systems statistical analysis 
based upon the equation in partial derivatives determining the 
first probability distribution of system outputs has been develop¬ 
ed 21 . 

Besides the general approximate methods of statistical non¬ 
linear systems analysis outlined above, other methods have been 
applied to solve some special problems. We shall confine our¬ 
selves to mentioning the papers by Feldbaum 22 ’ 23 and by Per- 
vozvansky 24 ’ 25> also n » Ch - 4 ), who have applied the theory of 
Markov chains to study the process of extremum seeking in 
sampled-data systems of optimalized control, and papers by 
Krasovsky, who has obtained the approximate differential 
equations for the expectations of the coordinates of continuous 
systems with extremal parameters adjustment 26-28 . 


The Precise Statistical Theory of Systems Reducible to Linear 


The statistical theory of linear systems is also applicable to 
certain non-linear systems which form a class of systems re¬ 
ducible to linear 1 ’ Sect * 106,107 .' This class contains, in particular, 
all systems obtained by cascading and joining in parallel linear 
systems, multipliers, devices for raising in various degrees and 
certain types of functional devices, provided that non-linear 
components are not contained in feedback loops. 

A system is called reducible to linear (in the statistical sense) 
if its input-output relation is of the form 



*t 
«, tc 


g(t,x 1 ,...,t k )<p(X(x 1 ),...,X(x k ),x u 
• ••>**, 0dTi,...,dT i 


(34) 


where g (t, x x ,..., xjc) is a weighting functions matrix, and 

cp (X (Tj), ...,X(x k ), Tj. T/c, t) is a definite vector function of 

the values of input vector XJj) corresponding to time instants 
r l9 xjc, which may also depend on r l9 ..., r j Ci t. 

Putting 

U (t, T l9 ...,T fc ) = <p(A (u)j •••? 2C (x k ) 9 T l5 ..., Tfc, t ) (35) 


the output vector of the system Y ( t ) is expressed as a result of 
a linear transform of the random vector function U : 


Y(t)= j ... g(t,x u ...,x k )U(t,x 1 ,...,x k )dx 1 ,.:.,dx k 

•I to J to - (36) 

Knowing 2 ^-dimensional probability density of the input 
vector (i.e. the joint probability density of values of all com¬ 
ponents of the vector X (t) 9 corresponding to arbitrary time 
instants r l9 ..., r*, r\ 9 ..., r' 7c ), the expectation and the correlation 
function of the vector random function U is defined by the well- 
known formulae of the probability theory, after which the ex¬ 
pectation and the correlation function of the output vector of 
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the system Y ( t ) may be determined by the methods considered 
in the first section. 

Any system whose output vector is a functional polynomial 
(i.e. a truncated functional Taylor series) with respect to com¬ 
ponents of its input vector is evidently reducible to linear 29 . 

Theory of certain special classes of systems reducible to 
linear has been developed by Zadeh 30 * 31 , Lubbock 32 - 33 and 
Prasad 34 . The general theory of systems reducible to linear has 
been given elsewhere 1 * 5 . 

As any system may be replaced (theoretically at least) by a 
sufficiently near system reducible to linear 30 * S1 , the statistical 
theory of systems reducible to linear may be used for approxi¬ 
mate evaluation of the accuracy of any non-linear systems. 

To apply the theory of systems reducible to linear to statis¬ 
tical analysis of linear systems with randomly varying dynamic 
characteristics it is sufficient to consider the matrix of weighting 
functions of a linear system G(t,r) as a random function 1 * Sect * 98 . 

Another somewhat less general approach to the statistical 
theory of non-linear systems has been given by Wiener 35 . 


The Precise Statistical Theory of One Class of Non-linear 
Systems Based upon the Theory of Markov Processes 

Besides considering in the previous section systems reducible 
to linear, the precise statistical analysis is now available only for 
such systems, which are described by ordinary differential equa¬ 
tions and inputs of which are either white noises or are expressed 
in terms of white noises by means of ordinary differential equa¬ 
tions of finite order (random disturbances, which are stationary 
random functions with rational spectral densities are an example). 

More precisely, the class of non-linear systems, which can be 
treated by the precise statistical theory based upon the theory 
of Markov random processes, is defined as the class of all 
systems described by systems of ordinary differential equations 
of the finite order of the form 

X = <p (t, X) + xp (t 9 X)V(t) (37) 

where X is the vector of coordinates of the system, the com¬ 
ponents of which are all the outputs of the system, all the neces¬ 
sary auxiliary variables and those of the system inputs which are 
not white noises, V ( t ) is the vector random function whose 
components are white noises in a strict sense*, <p (/, A") is a vector 
function of the time t and the vector U, ip (t, X) is a matrix whose 
elements may be functions of t and X. 

The vector of system coordinates X defined by the system of 
differential equations (37), subject to conditions indicated, re¬ 
presents a Markov random process, and its first probability 
distribution is determined by the well-known equations of the 
theory of Markov processes. 

The theory of Markov random processes was used at first 
for analysing dynamic systems accuracy by Andronov, Vitt 
and Pontryagin 36 . Further development of the statistical theory 
of non-linear systems, based upon the theory of Markov pro¬ 
cesses, has been given by Barrett 37 , Pugachev 38 * 39 , Hazen 40 * 41 
and others. 

As shown 38 * 39 , the first characteristic function ^ (X; t) of 


* ^^ite noises in a strict sense are such random functions whose values for 
different values of the argument are independent and whose correlation and cross - 
correlation functions have as a factor (5-function of the difference of arguments. 


the vector of system coordinates X (/) defined by (37) ist deter¬ 
mined by the linear integro-differential equation 


6gi (A; 0 1 


8 t 


(2n)"J - 


dx 


J (A-/x) x 


<P(X;t,x)g 1 (ii;t)dp (38) 

where n is the number of components of the vector X (/), x, X 
and n are ^-dimensional vector arguments, (X—fj)x is the scalar 
product of the vectors X—jli and x, and & (X; t, x) is the value 
at At = + 0 of the derivative with respect to At of the condition¬ 
al characteristic function of the increment of the process X ( t ) 
during the time At, when its value x at t is given: 


$(X;t,x) = iX(p (t, x) -j -x(X;t,x) 


x(X;t, x) = 


iff f f M 

lim ~r\M zxpiiX' 

v- + 0 A t{ L { Jt 


xp (t, x) V (t) dr 


(39) 

(40) 


(the point denotes the scalar product). 

Under the same conditions the first probability density 
fi(x; t ) of the process X (/) is determined by the linear integro- 
differential equation 39 


0 /i(*;O 1 


dt 


(2nY\ 


dX 


otA (£ x) sj*. / i . 




(41) 

In the special case, in which the first and the second moments 
of the random vector 

't + At 

z= I V (t) dr (42) 


are of order At, and all higher moments are infinitesimal of 
higher orders, the eqn (41) is reduced to the well-known Fokker- 
Plank-Kolmogorov equation. 

In another special case, where the function <P(X; t,x) is a 
polynomial with respect to components of the vector x 9 eqn (38), 
defining the characteristic function of the system coordinates, is 
reduced to the linear equation in partial derivatives 


dgiU;0 

dt 


= # X;t, 


idX 




(43) 


where d/dX is the vector gradient in the space of the vector X. 

In the special case of a linear system, eqn (43) is of the first 
order and is easily integrated by the well-known standard pro¬ 
cedure. 

The solving of eqns (38) and (41) for g ± (X; t) and (*; t) 
(in respective special cases eqn (43) or Fokker-Plank-Kolmo- 
gorov equation) may be accomplished analytically only in the 
simplest problems. Thus, only numerical solution by means of 
digital computers is available in a great majority of problems. 
But the calculations required are so tedious, that it is possible 
to accomplish them with the aid of electronic digital computers, 
only when the order n of the system of differential equations (37) 
(i.e. the number of degrees of freedom of the system) isn’t too 
high. The volume of calculations involved and the necessary 
storage capacity of computers increase rapidly with the increas¬ 
ing of the order of differential equations. That is the reason why 
the theory of Markov random processes doesn’t yet find suf¬ 
ficiently wide applications in automatic control problems. 
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The Problem of Finding an Adequate Dynamic Model of a Con¬ 
trolled Process or a Plant 

One of the most important problems of the statistical theory 
of automatic control systems is the determining of dynamic 
characteristics of systems and plants from normal operating 
records or, more precisely, finding adequate models of systems 
and plants. To solve this problem it is necessary to find, by a 
suitable statistical processing of normal operating records, the 
correlation function of the input vector to the system under 
study K x 0, t') and the cross-correlation function of the output 
and input vectors K yx (t,t'). Then the weighting functions 
matrix of the linear system or plant is determined by solving the 
set of linear integral equations of the form of eqn (28) 42 > 43 . 

This method enables us to determine dynamic characteristics 
of linear systems subject to noise effects with better accuracy, 
than the usual methods of determining responses to standard 
types of inputs. 

The eqn (28) also defines the weighting functions matrix of 
the linear system yielding the best approach, in the mean square 
error sense, to a given non-linear system (i.e. the best linear 
model of a given non-linear system). 


Methods of Determining Optimal Linear Systems 

The problem of determining characteristics of systems from 
normal operating records is a special case of more general prob¬ 
lems of the statistical theory of optimal systems. This theory 
enables us to find optimal mathematical operations over inputs 
providing the best attainable accuracy for given statistics of 
signals and noises. Thus, the theory of optimal systems affords 
finding the theoretical limit of the accuracy of a system of a given 
destination. Comparing with this theoretical limit the accuracy 
of the system to be designed, the engineer can estimate the ac¬ 
curacy of this system and provide the nearness of the system 
accuracy to the optimal one. 

The general problem of the statistical theory of optimal 
systems may be stated in the following way: it is necessary to 
find the system operator A in such a manner, that for a given 
input Z (4 representing a signal distorted by noise, the system 
output W* (/) = AZ (t) be as near as possible to a desired out¬ 
put W(t ), the statistics of the input and desired output being 
given. This problem covers all problems of the determining of 
optimal systems and optimal information processing algorithms, 
arising in automation, as well as a variety of analogous problems, 
arising in other branches of science and engineering. Such are, 
for example, the problems of optimal detection of various signals 
in noises, arising in radio-engineering, seismology and other 
branches of engineering; the problem of the best reproduction 
of the sound and TV picture; the determining and the extra¬ 
polation of the elements of trajectories of satelites and cosmic 
rockets from the observed data, etc. 

To estimate the nearness of the system output W* to the 
desired output W, various criteria are used depending on the 
destination of the system. Almost all practically employed 
criteria admit representation in the form of a condition of mini¬ 
mum average risk: 

q = M [l(W, W*)~]= min (44) 

where / is a certain function (or functional) of desired and actual 
systems outputs, generally called the loss function. The criteria 


of the form (44) are generally called Bayesian criteria. The opti¬ 
mal algorithm of the system, processing output W* for a given 
input Z in accordance with the criterion (44), is called Bayesian 
decision. 

For automatic control systems the criterion of the minimum 
of the mean square error is most widely used, to which corre¬ 
sponds the quadratic loss-function 

l(W,W*)=(W*-W) 2 (45) 

Although this criterion is the simplest from a mathematical 
viewpoint, it isn’t adequate in many problems. Other criteria 
must then be used. So, for example, in the problem of signal 
detection in noise the criterion of the minimum probability of 
the error is expedient, sometimes with certain additional con¬ 
ditions. This criterion admits the representation in the form of 
eqn (44), where the loss function / is zero for the correct decision 
and has definite positive values for errors of two kinds (the 
decision that the signal is absent, when it is present, and the 
decision that the signal is present, when only the noise enters the 
system). In more complex situations the loss-function may de¬ 
pend not only on the present values of the desired and actual 
outputs, but may be a functional of W and W*. 

In many practical problems a signal contained in the system 
input Z has a regular component, which is a known function of 
time and the finite number of parameters U lf ..., U N - These 
parameters are random variables or unknown non-random quan¬ 
tities which may have any value. The desired output is then the 
result of a given transform of the signal contained in the input Z. 

If the regular part of the signal is a linear function of para¬ 
meters U lt ..U N , and the desired output represents a result of 
a given linear transform of the signal, then 


Z(0= Z U r cp r (t) + X (t) 


(46) 


W(t)= Z U r ip r (t)+Y(t) 


(47) 


where <p r (0. Vr (0 (r = 1,..., N) are known functions of time, 
X (0 is the sum of the irregular part of the signal and the noise, 
and Y ( t ) is the result of a given linear transform of the irregular 
part of the signal. The problem of finding the optimal linear 
system, using the minimum mean square error criterion, is then 
reduced to solving N + 1 systems of linear integral equations 


g (0> (t, t) K x (t, a) dr = K yx (t, a) (t-T<a< t) 


t-T 


g ir \t,T)K x (T,Cl)dT = q> r (<T) 

t-T 

(t-T<a<t;r={, ..., N) 
and the system of linear algebraic equations 

Y + = (jp=l, N) 

q=l 

where Cpq and C $ 0 are defined by 

't 


(48) 

(49) 

(50) 




g (q) (f, t) cp p (t )dr = 


t-T 


g ip) (Ur)q) q (T)dT (51) 


t-T 
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CpO~ 


g (0) (t, x) (p (t) dr 


J t-1 


K yx (*> T) g (P) (t } T) dl 


Op=i 9 ...,jv) 


(52) 


and C#g, are the elements of the matrix, inverse with respect to 
the matrix of second moments of the random variables U 19 ...,Un : 


y M ==M[i7 p tg (53) 

The product of vector functions (f, t) and ^ (r) in (51) is 
the scalar product. 

If U r is a non-random unknown quantity which may have 
any value, then all Cpq, for which either p or q is equal to r, are 
equal to zero. 

After finding the weighting functions matrix g<°), the vector 
functions gQ-\ ..gW and the vectors 2 l5 ..2#, the weighting 
functions of the optimal system are given by 


Zki(t,*) = g ( M ) (t, t)+ I X rh gj r) (t,x) 


r = i 


(54) 


The mean square of the error of the kth output of the optimal 
system is given by 


Vh min & yh (t) 


-zf 


ShV (t, t) Ky hXl (t , x) dx 


N 

4” X! ^rh IjPrh (f) (£>o)/J 

r-1 


(55) 


In the special case, where the signal has no irregular com¬ 
ponent, Y (t) = 0, D yn (t) = 0, K yx (t 9 x) = 0, *W> (/, r) ss 0, 

Ci ° - ... Cn o = 0, and eqns (54) and (55) are accordingly 

simplified 1 » Secfc * 120 * 122 - 124 * 12 $ m 

Andreev has shown that the problem of finding the optimal 
system, using a more general criterion of the extremum of any 
pre-assigned function of the expectation and the variance of the 
system error, is reduced to equations of the same type, as 
above 44 ' 46 . 

Thus, the problem of determining the optimal linear system, 
using the mean square error criterion or a more general criterion 
of extremum of a given function of the expectation and the 
variance of the system error, is reduced to solving the system of 
linear integral equations of the form 



g (t 9 t) K x (t, a) dr = / (f, a) 




(56) 


where f(t, a) is a known matrix, and g(t, x) is the unknown 
matrix of weighting functions. These equations define the weight¬ 
ing functions of a continuous linear system in the general case. 
Thus, the optimal system among all linear systems is a contin¬ 
uous linear system in general. To find the optimal sampled-data 
linear system, it is necessary to search its weighting functions in 
the form of linear combinations of (5-functions. Eqn (56) is then 
reduced to a set of linear algebraic equations. 

Eqn (28), encountered in the statistical linearization problem 
as well as in the problem of estimating the dynamic character¬ 
istics of systems from normal operating records, is evidently a 
special case of eqn (56). 

All known methods of solving the equations of the form of 
eqn (56) are in fact based upon the method of canonical re¬ 
presentations of random functions. The method of integral 


canonical representations yields the solution of such equations 
in a closed form in all cases, where an integral canonical re¬ 
presentation of the random function Xcan be found 1 ’ SecL 128 “ 133 . 
The method of canonical expansions provides an approximate 
solution in the form of truncated series in all cases, as any random 
function can always be represented by a canonical expansion in 
an infinite number of ways 1 ’ Sect - 134 ~* 137 . 

To solve eqn (56) by the integral canonical representations 
method, when the observation interval T is infinite, it is necessary 
to find a linear system transforming the vector random function 
Xto a corresponding number of uncorrelated white noises. The 
weighting functions matrix w~ (/, r) of such a system have been 
found, the solution of eqn (56) for T = oo is given by 

g(t,r)= J dl j j (t, a) w — (2, <j) G 1 (2) w”(yi, r)d <7 (57) 

where G (2) is the diagonal matrix of intensities of the com¬ 
ponents of the vector white noise, to which the vector random 
function X is transformed by the system with the weighting 
functions matrix w~ (t. r). 

In the special case, where the correlation function K x (r, a) 
depends only on the difference x — a (i.e. the vector random 
function X is stationary), and the right member of eqn (56) 
f (^> °0 is dependent only on the difference t — a [for instance, 
a cross-correlation function of two stationary and stationarily 
correlated random functions Y (a desired stationary vector of 
system outputs) and X (a stationary input vector)], f(t, a) 
= h(t — or), eqn (57) yields the known formula of Wiener’s 
theory of smoothing, interpolation and extrapolation of station¬ 
ary random processes 1 ’ Sect * 129 * and 48 > 49 ; 




V (p) $ 1 (i» <2> 1 (^l) e CfA,_A > e djit(58) 


where W (A) is the matrix of transfer functions of the optimal 
stationary linear system, 'ip (w) is given by 


V>0)=Tj_ hi^e-^dri (59) 

and &- 1 (2) is the square matrix of transfer functions of the 
stationary stable minimal-phase system transforming the vector 
random function X to the vector white noise with uncorrelated 
components having unit spectral densities (i.e. the intensities 
equal to 2 n). The special case of Wiener’s formula for a single¬ 
input and single-output system has been generalized by Booton 
for an arbitrary function/(t, cr) 50 . 

The general formula (57) may also be used to find the solution 
of eqn (56) in the case of an arbitrary finite observation interval 
T. For this purpose it is sufficient to let g (t 9 r) = 0 for r < t — T 
in (57), and to assume f(t 9 r) as unknown in the interval — oo 
< r < t—T. This yields simultaneous equations determining 
the matrix g (t, x) in the interval t — T <; t < t and the 
matrix f(t, x) in the interval — oo < x < t — T. These 
equations are easily solved in the case, where the vector random 
function X is stationary and has a rational matrix of spectral 
and cross-spectral densities (Zadeh and Ragazzini’s case 51 and 
its generalization 48 ), and in a more general case, where the vector 
random function X can be expressed in terms of a vector white 
noise with a respective number of uncorrelated components by 
simultaneous linear differential equations with sufficiently smooth 
Coefficients 1 ’ ® ec ^* 13 ®-133; 48, and 2 , Sect. 8-5 
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To solve eqn (56) by the canonical expansions method it is 
necessary to find a canonical expansion of the vector random 
function X (t) of the form of eqn (3) in the observation interval 
t—T < x < t. Then vector functions a v (r) will be found, 
satisfying, together with the vector coordinate functions x v (r), 
the equations 

x v (va ll (x)d-c=5 vll 

Jf-T 

1 f* 

x v( t ) = 7T I K x {x,G)a v (o)&G (60) 

Then the solution of eqn (56) is defined by 1 ’ Sect * 134 ~ 137 


g(.t,r) = 


y gvOO 
Zj JT) 

v 


f(t,<r)a v (<j) da 

t-T 


(61) 


where the point means the product of a vector column by a 
vector row. Eqn (61) represents the solution of eqn (56) in all 
cases, where the right member of eqn (56) can be expressed in 
the observation interval t — T < x < r by the coordinate-func¬ 
tions expansion 


/(<> *)=£ *»(*)• 
V 


fit, a)a v (<j) d(T 

t-T 


where the point has the same meaning as in eqn (61). This con¬ 
dition (which is equivalent to the condition of the completeness 
of the set of coordinate functions x v (x) in the observation inter¬ 
val t — T < x < t) is generally fulfilled in practical problems. 

Eqn (61), in particular, may serve for determining optimal 
sampled-data linear systems. The random function X should be 
expressed by the expansion (3) for all the instants of application 
of inputs to the system in this case. The functions a v (t) being 
linear combinations of (5-functions in this case, the function 
g ( t,x ) determined by eqn (61) will also be a linear combination 
of (5-functions, i.e. it will represent the weighting functions matrix 
of a sampled-data linear system. For a finite number of instants 
of the application of inputs to a system, eqn (61) yields the 
precise solution of corresponding simultaneous linear algebraic 
equations determining the optimal sampled-data linear system. 

The above outlined methods of finding optimal linear systems 
cover practically all problems of this kind encountered in auto¬ 
mation, radio engineering, and in other branches of sciences and 
engineering. In its general form the canonical representations 
method is also applicable to problems in which the observed 
signal (input vector) Z, as well as the desired and actual outputs 
W , tV* are random functions of several variables. Eqn (61), in 
particular, is also valid in this case, the variables t, r, a being 
vectors, and the integrals being multiple and extended over the 
corresponding domain of the space of the vectors x and a (the 
observation domain). 

The common deficiency of all methods outlined above is that 
they determine only overall system weighting functions and 
provide no solution of optimal systems synthesis problem, when 
some invariable components of the system (plants for instance) 
are given. Owing to this, various attempts have been made to 
find other methods allowing a direct optimal linear systems 
synthesis. 

The most interesting and effective results in this direction 
have been obtained by Kalman and Bucy 52 > 53 . Their approach 
yields directly differential equations of the optimal system. Un¬ 


fortunately this method is restricted to problems, where the inter¬ 
ferences are white noises, and the signal is formed from white 
noises passing through a certain linear system during the obser¬ 
vation interval. Owing to this, Kalman and Bucy’s method can 
be used only for the synthesis of linear systems with infinite 
memory and with white noises as interferences. 

Another interesting approach to the problem of the direct 
optimal linear system synthesis is formed by attempts to extend 
the theory of analytical design of controllers to systems with 
random inputs 54 . 

The statistical problem of finding optimal control was at first 
stated and solved under certain assumptions by Pontryagin 55 . 
This theory has been developed further by Krasovsky 56 ’ 57 . Yet 
the common deficiency of all papers, in which the theory of 
analytical design of controllers is extended to systems with 
random inputs, is the assumption that random disturbances are 
directly measurable, whereas neither signals nor noises can be 
measured separately in general. This fact being taken into ac¬ 
count in Zaytsev’s papers 58 , a certain generalization of Wiener’s 
problem of finding optimal stationary linear system has been 
obtained as a result. 

The statistical theory of optimal linear systems is also ap¬ 
plicable to systems reducible to linear. An effective solution of 
the problem of finding the optimal non-linear system among 
systems reducible to linear was at first given in 1956 59 . General 
methods of solution of this problem, stated in all possible ways, 
have also been given in 1 * Sect ‘ 138,139, and 5 . The equation determin¬ 
ing the optimal system for one of the possible statements of the 
problem was previously obtained by Zadeh 30 * 81 . The solutions 
of some concrete problems of finding optimal systems reducible 
to linear have been given by Lubbock 32 ’ 33 and Prasad 34 . 


The Application of Statistical Decision Theory to Problems of 
Finding Optimal Systems and Algorithms of Information Pro¬ 
cessing 

As stated in the Introduction, the optimal statistical infor¬ 
mation processing is of great importance in the design of com¬ 
plex systems containing many elements. The general problem 
of the statistical theory of optimal systems, stated in the begin¬ 
ning of the preceding section, covers all problems of finding 
optimal algorithms of automatic systems taken overall and of 
their various components as well. Hence the methods of solving 
this problem may serve for estimating the potential accuracy 
and potential interference-stability of overall automatic systems 
and for designing optimal components of automatic systems 
as well. In particular, they may be used in the design of systems 
for the detection and extraction of signals in noises. 

The general problem of the theory of optimal systems, stated 
in the beginning of the preceding section, is a typical mathe¬ 
matical problem of the statistical decision theory. Hence the 
natural theoretical basis for the solution of problems of this kind 
is given by the statistical decision theory initiated by Wald 60 
and primarily used for solving problems of detection and extrac¬ 
tion of signals in noises by Middleton and van Meter 61 " 63 . The 
effective general method of solving a wide class of problems of 
the statistical decision theory, applicable to the majority of prob¬ 
lems arising in automatic control theory, has been 1 ’ Sect - 141 ~ 145, 

smd 64-70 developed. 

Let S be an input signal to the system or its component to 
be designed. This signal is corrupted by noises, owing to which 
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the input Z enters the system instead of S. The ensemble of 
possible realizations of the signal S is characterized by a priori 
probability density / ( s ). 

The effect of noises is that the input Z is random for each 
realization s of the signal 3. Hence the effect of noises is charac¬ 
terized by the conditional probability density / x (z|5) of the input 
Z with respect to the signal S. 

The desired output of the system (or its component) to be 
designed W is usually the result of a certain pre-assigned trans¬ 
form of the signal S . The statistical decision theory admits, in 
general, that this transform may be random and consequently 
may be defined by the conditional probability density / 2 (w\s) of 
the desired output W with respect to the signal 3. 

The optimal algorithm of the system (or its component) to 
be designed may also be random in the general case. So it will 
be completely determined, if the conditional probability density 
<5 (w*|z) of the output W* with respect to the input Z is found. 
The probability density (5 (w*\z) or the corresponding probability 
measure is called the statistical decision function. 

We emphasize, that the desired output W is dependent on 
the signal S only and is independent of noises. Hence / 2 (w\s) is 
identically equal to the conditional probability density of W 
with respect to 3 and Z:/ 2 (w|s) = / 3 (w|j, z). Similarly the actual 
output W* is dependent only on the input Z and cannot depend 
on 3 and W. Hence the statistical decision function (5 (w*|z) is 
identically equal to the conditional probability density of W* 
with respect to S , Z and W\ 6 (w*|z) = ( w*\s , z, w). 

Using these facts, the joint probability density of 3, Z, W 
and W* is given by 

/o (s, Z, W, w*)=/(s)/ 1 (z\s)f 2 (w|s) 5 (w*|z) (62) 

where probability densities f f l9 / 2 are known, and S is to be 
determined from the condition of the minimum of the average 
risk (44), which is evidently dependent on S. Putting this in 
evidence and using (62), we obtain the following expression for 
the average risk: 


q(8)=m[i(w, w*y ] 


— JJJJ l ( w , w*)/ (sj/j (z\s)f 2 (w|s) 8 (w*|z) dsdzdwdw* 

u . - • (63) 

where the integration is extended over the domain of all possible 
values of the signals 3, Z, W, W*. 

Noting that s . 

f(s)f 1 (z\s) = p(z)p 1 (s\z) (64) 

where 

P( z ) = J/(s)/i(z|s)ds (65) 


is the unconditional probability density of the input Z, and 
Pi C*|z) is the conditional probability density of the signal S with 
respect to Z, we may write eqn (63) in the form 


8 (<5) = J p (z) dz J q (w*, z) 8 (w*| z) dw* (66) 

where 

q (w\ z)= JJ l(w, w*) p x (s|z)/ 2 (w|s) dsdw (67) 

The function q (w*, z) represents the average conditional risk , 
corresponding to definite realizations w* 9 z of signals W*, Z. 

It is evident, that if for each realization z of the input Z there 
exists a unique value w* of the variable w*, for which 

Q (w*, z )=inf q (w * 9 z) = Min q (w * 9 z) (68) 


then the Bayesian decision minimizing the average risk q (<5) is 
the Dirac (5-function (i.e. the probability distribution condensed 

in a single point vv*): 


<5 (w*jz) = <5 (w* — w*) 

(69) 

The value vv * is evidently dependent on z: 


w* = Az 

(70) 


where A is a definite operator. 

Thus the minimizing of the average risk yields the unique 
deterministic operator of the optimal system 


W* = AZ (71) 

provided that for each realization z of the signal Z there exists 
a unique value w* satisfying (68). 

In the case where the unique vv* = Az satisfying (68) doesn’t 
exist, the optimal system operator (optimal algorithm of infor¬ 
mation processing) may be random. 

The above propositions of the statistical decision theory can 
evidently be directly applied only when the signals S , Z, W and 
W* are finite-dimensional vectors. Hence they are immediately 
applicable to problems of sampled-data systems design only. It 
is certainly possible to rewrite them for the case where S, Z, W 
and W* are random elements in any spaces, particularly, func¬ 
tional, using corresponding probability measures instead of 
probability densities. But the direct application of the theory 
would be impracticable in this case. Therefore another way of 
the extension of the above results to continuous systems is 
preferable, that is, the passing to the limit when the number of 
dimensions becomes infinite. 

The limiting process in the preceding formulas, when the 
number of dimensions becomes infinite, may be accomplished 
in two different ways. 

The first way is to replace all functions of continuous argu¬ 
ments by their values at sampled points and then to increase 
indefinitely the number of these points, all distances between the 
consecutive points becoming zero. This way has been used by 
many authors, particularly by Laning and Battin 2 > Sect - 8 ' (i and 
Middleton 63 . But this approach leads to considerable mathe¬ 
matical difficulties, owing to which the way of proving rigorously 
the existence of all limits encountered has not yet been found. 
Hence the passing to the limit from the sampled-data case to the 
continuous one remains, in the meanwhile, purely heuristic and 
is not strictly founded. 

Another way is to express all random functions of continu¬ 
ous arguments in terms of random parameters and to pass to 
the limit from a finite set of random parameters to a denumerable 
one. This approach enabled the rigorous theory to be developed 
for determining operators of deterministic optimal systems under 
rather general conditions 64-70 , the random functions being ex¬ 
pressed in terms of random parameters with the aid of canonical 
expansions of the form of eqn (3). 

We shall confine ourselves here to outlining the method 
derived 64-70 for the simplest case, which covers nevertheless, a 
sufficiently wide variety of practical problems. 

In many practical problems the input signal S doesn’t contain 
irregular components and is the known function (non-linear in 
general) of the parameters U l9 ..., U n and of time t, and the 
noise is additive and normally distributed. The input Z and the 
desired output W are expressed by 
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Z(t) = cp(t,U) + X(t), W(t) = ip(f, U) (72) 

in such cases, U being the vector signal parameter with com- 
^ ponents f/ 3 ,.. Un , <p and ip are given functions of f and U , and 
Z (0 is a normally distributed noise independent of the vector 
U. Then the method derived in 64 " 70 yields the following general 
result (see 12 » Sect ' 16,1 ): the optimum is attained by such a system 
which, receiving at the input Z, is processing output by mini¬ 
mizing the integral 


/(fF*,Z) = 


l(xp(t,u ), W *) 

— 00 1 

exp [L (u) Z (t) —(m)}/(w ) di/ (73) 


/(«) being the joint probability density of all random compo¬ 
nents of U(f(u)= 1, when all the components of U are un¬ 
known non-random quantities which can assume any values), 
L (u) is the linear operator 


L(w)Z(t) = 


t 

g(t,T,u) Z(t)dT 


t-T 


(74) 


whose vector of weighting functions g ( t , r, «) is defined by 
simultaneous linear integral equations of the form of eqn (56) 


g(t,r,u)K x (t,a)dT = (p(a-,u) ( t—T<a<t ) (75) 

J t-T 

and the function /7 (//) is defined by 


P(u)==L(u)(p(T, u) = 


g(f,T,u)<p(i:, w)di 

t-T 


(76) 


The above relations determine the algorithms of optimal 
systems of various types, depending on the choice of the loss 
function (or functional) and the way in which function ip is re¬ 
lated to cp. In particular, they yield the general solution of signal 
detection problem for signals depending on a finite number 
of unknown parameters 1 ’ Sect 141 ’ 14a . The procedure outlined 
above proves also applicable to certain cases of infinite number 
of signal parameters U r l * Sect ’ 144, and 70 - 

In the case, where Z and W are linear functions of signal 
parameters C/ ls Ui y, the probability density f(u) is normal 
(or f(u) = 1), and the loss-function (functional) is dependent 
on the system error W* — W only, the optimal system, process¬ 
ing in accordance with eqns (73)—(76) is linear 1 ’ Bect - 143, 

The above method is easily generalized and is also apt for 
much more complicated situations. In particular, it is applicable, 
when the input signal Z contains an additive irregular compo¬ 
nent, or when the irregular part of the input signal and 
the noise are not additive, provided that the input Z is reduc¬ 
ible to the form of the first eqn (72) by a known non-linear 
transform 1 ’ Sect 144 . This general method allows the solution 
of various problems of detection, discerning and extraction of 
signals arising in automatic control and radio engineering, and 
also of analogous problems in other branches of sciences and 
engineering. This method is applicable to single-dimensional 
systems and to multi-dimensional systems with an arbitrary 
number of inputs and outputs as well, the functions Z, W and 
W* being vectors in the latter case. Applying the method of 
canonical expansions to solve eqn (75) the above method is also 
applicable when Z, W and W* are functions not only of time, 


but also of other arguments, for example, of coordinates of 
points in the space. 

Optimal systems algorithms defined by the above method 
may be used in real automatic systems containing digital com¬ 
puters (sufficiently high-speed). In particular, these algorithms 
may be used in the design of ‘self-learning’ systems automatically 
studying the statistics of inputs and processing the control pro¬ 
gramme accordingly 66 . 

The mathematical basis for simulating the processes of learn¬ 
ing and experience storing in automatic systems has been devel¬ 
oped with the aid of the statistical decision theory by the group 
of Czechoslovak mathematicians with the late A. Shpachek at 
the head 71 " 74 . 

The peculiarity of feedback systems is that the input of the 
part of a system involved in a feedback loop is dependent on its 
output. The dynamical properties of any system (or any com¬ 
ponent of a system) are characterized in the general case by the 
conditional probability density of output for a given input. If 
one of the inputs of a certain part of the system is dependent on 
its output, it is impossible to define the conditional probability 
density of output with respect to input for this part of the system. 
Consequently it is impossible to define directly the conditional 
probability density of output for a given input for the system 
component involved in a feedback loop. Hence the statistical 
decision theory can be applied in the design of systems compo¬ 
nents involved in feedback loops, only if some additional as¬ 
sumptions are made, or an iterative process must be used. In 
particular, the statistical decision theory may be directly used for 
sampled-data systems with delayed feedbacks, the inputs of 
which at any time are dependent only on the values of 
respective outputs at past instants. The statistical decision 
theory yields in this case multi-stage optimal decisions defining 
the optimal algorithms of sampled-data systems with delayed 
feedbacks. It is by this method that Feldbaum has developed 
the so called dual control theory, using the statistical decision 
theory 75 ” 78 . 

Conclusions 

We see, that statistical methods in automatic control are now 
well developed and represent a powerful tool for the design and 
study of automatic systems and for the solution of automatic 
control problems. Voluminous literature has been involved in 
this field. We have considered here only the most important basic 
problems of the statistical theory of automatic control, and many 
important works done in various countries by scientists of vari¬ 
ous nationalities have been mentioned. Nevertheless all papers 
mentioned here represent only a small part of scientific papers 
in the field of statistical methods in automatic control. 

In spite of the progress achieved, many statistical problems 
of the automatic control theory are not solved yet. Many prob¬ 
lems are waiting for their statement and solution. 

The basic problem of further development of the statistical 
theory in the near future is undoubtedly the creation of the 
statistical theory of complex systems and of methods for con¬ 
trolling complex systems having a statistical nature. 

Automation is now expanding to a constantly increasing 
number of fields of human activity. In particular, the works in 
the field of automatization of diagnostics in medicine and in the 
field of application of methods of automatic control theory to 
biological problems and to problems of other branches of sciences 
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as well have been started. Statistical methods of automatic 
ntrol theory will undoubtedly be useful in these fields and so 
will contribute to the progress of mankind. 
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A New Method to Derive the Describing Function of Certain 

Non-linear Transfer Systems 

R. LAUBER 


Summary 

Automatic control system components may often be described by non¬ 
linear differential equations with products or powers of the system 
variables. Besides the examples cited below, every adaptive control 
system contains products of variables. Another important class of 
control systems are those with periodic coefficients. To determine the 
stability of control systems of this kind, one is interested in the describ¬ 
ing function of its components. 

To derive the describing function of such non-linear systems a 
method is presented, which is very similar to the well-known proce¬ 
dures to calculate frequency-responses of linear systems. It yields a 
system of algebraic equations with complex variables which is an 
implicit form of the describing function. The derivation of these equa¬ 
tions is shown in a general manner. 

To demonstrate the application of the method, several examples 
are considered. The first is a transfer system with a product of the input 
and output variable. Using all higher harmonics in the output, a 
describing function is derived which is an infinite continued fraction. 
A practical case of a system of this form is a nuclear reactor. The 
results for this second example are briefly demonstrated. In a third 
example the method is applied to a non-linear system with a square of 
the output variable. 

Sommaire 

En etudiant des asservissements non-lineaires on est conduit souvent a 
des systemes physiques represents non pas par unecaracteristique, mais 
par des equations differentielles contenant des produits ou des puis¬ 
sances des variables. Outre les exemples cites ce sont surtout les asser¬ 
vissements adaptifs qui contiennent des produits des variables. Pour 
determiner la stabilite des systemes de ce type une analyse sinusoi- 
dale (au lieu d’une analyse temporelle) est suggeree ici, tres semblable 
a la methode frequentielle qu’on emploie dans le cas des systemes 
lineaires. On obtient done un systeme d’equations complexes con- 
tenant la fonction de transfert sous forme implicite. 

L’application de la methode suggeree est expos6e sur plusieurs 
examples. Premierement un systeme simple caracterise par une 
equation differentielle contenant un produit du signal appliqu6 x e (t) 
et de la reponse indicielle x a (t) est calcule. La fonction de transfert 
implicite de ce systeme contenant toutes les harmoniques de la reponse 
forme des equations lineaires complexes, dont la solution explicite 
donne une fraction continue infinie. 

A titre d’exemple pratique et important d’un systeme de ce type, 
le reacteur nucleaire est brievement traite. Le troisieme exemple est 
un systeme physique represente par une equation differentielle non- 
lineaire contenant des puissances des variables. 

Zusammenfassung 

Die Bestandteile eines automatischen Regelsystems konnen oft durch 
nichtlineare Differentialgleichungen beschrieben werden, welche Pro- 
dukte oder Potenzen der Systemveranderlichen enthalten. AuBer in 
den am Ende angeflihrten Beispielen treten in jedem adaptierenden 
System Produkte der Veranderlichen auf. Um die Stabilitat solcher 
Anordnungen zu bestimmen, benotigt man die Beschreibungsfunktion 
der Bausteine. 

Zur Ermittlung der Beschreibungsfunktion derartiger nichtlinearer 
Systeme wird ein Verfahren vorgeschlagen, welches der bekannten 


Bestimmung der Frequenzgange linearer Systeme ahnlich ist Es lie- 
fert ein allgemeingultiges System algebraischer Gleichungen mit 
komplexen Veranderlichen, welches in impliziter Form die Be¬ 
schreibungsfunktion darstellt. 

Der Veranschaulichung dienen mehrere Beispiele. Das erste ist 
ein Ubertragungssystem, in dem das Produkt der Eingangs- und Aus¬ 
gangsgroBe auftritt. Unter Benutzung aller Oberschwingungen der 
AusgangsgroBe ergibt sich die Beschreibungsfunktion als unendlicher 
Kettenbruch. Ein der Praxis entnommenes Beispiel, fur welches einige 
Ergebnisse mitgeteilt werden, ist der Kernreaktor. Zum AbschluB 
wird die Anwendung des Verfahrens auf ein nichtlineares System ge- 
zeigt, in dem das Quadrat der AusgangsgroBe von Bedeutung ist. 


Introduction 

Instabilities in feedback control systems will mostly cause 
oscillations. Linear systems are able to oscillate sinusoidally; 
non-linear systems may exhibit sustained periodic oscillations. 
Frequently these will be nearly sinusoidal, which means that 
there exists a fundamental component predominant to higher 
harmonics. If this holds, the stability of the non-linear con¬ 
trol system can be investigated by considering only one or at 
least very few harmonics. The well-known frequency-response 
methods are then applicable, provided the describing function 
is used in the same way as the frequency response function of a 
linear component. 

In its usual form the describing function describes the behav¬ 
iour of a non-linear element for harmonic inputs. Assuming a 
pure harmonic variable 

x e ( t ) := $ e i sin cot (I) 

at the input which may cause a periodic output (see Figure 1) 



x«(O=Aisin(c0t+<j6 1 )+£ a2 sin(2a)i+<£ 2 ) + ... (2) 

the describing function is defined as the ratio 

iV = —e- w ’ 1 (3) 

*et 

This definition can be generalized by assuming more than 
one harmonic at the input of the non-linear component 
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x e (t)= Z % ev sin vcat (4) 

v = 0 

producing a periodic output 

00 

*<,(*) = Z x av sin(va>t + (j) v ) (5) 

v = 0 

If only the fundamentals of input and output are again com¬ 
pared the same definition (3) as shown above is obtained. 

It should be mentioned though, that the consideration of 
higher harmonics in the input function x e ( t ) will be necessary 
only in very few cases; on the other hand, for many non-linear 
systems a term of order zero, the d.c. component x e0 , must be 
taken into account. 

The calculation of the describing function is done usually in 
the time domain by a Fourier transform of the periodic output 
function x a (t). This method is quite effective for the most 
important non-linearities described 1 by a function x a (t) 

= /[*. m 

Nevertheless, some non-linear control system components 
are described by an implicit function g [x e (f) 9 x a (*)] = 0 or 
by a differential equation D [x e ( t ), x e (t), x a ( t ), ...] = 0. 

Here the explicit evaluation of the periodic output x a ( t ) for 
harmonic input (t) and thus the calculation of the de¬ 
scribing function might be difficult. Methods are known 
to calculate describing functions for such systems in the time 
domain under the assumption that all higher harmonics in the 
output are zero, which may be the cause of serious errors. 

Extending these approaches 2 ’ 3 a new method will be 
shown here for an evaluation in the frequency domain. Using 
complex methods analogous to those applied to derive fre¬ 
quency responses of linear systems, describing functions can 
frequently be derived in analytic form, if the non-linearities in¬ 
volved are products or powers of the system variables. 


General Method to Determine Describing Functions in the Fre¬ 
quency Domain 

A non-linear system may be described by a differential 
equation 

D [x. (0, x e (t),...,x a (t),x a (t),...']=0 (6) 

The input variable ( t ) may consist of one or several harmonics: 

m 

x e (t) = M E (7) 

0 

If the output will be periodic, it can be approximated by a sum 
of harmonics 

x a {i) = m E x av e JV0>t (8) 

v = 0 

With these expressions (7) and (8) introduced, the differential 
equation (6) will be fulfilled the better, the more terms in sum 
(8) have been considered. On the other hand, a limited number of 
terms will give satisfactory results, as it is the core of the describ¬ 
ing function approach to ‘balance’ harmonic terms rather than 
to solve exactly the differential equation. In the equation 


derived by inserting the expressions (7) and (8), this principle 
corresponds to a comparison of coefficients of the terms of the 
same frequency. 

Such a comparison is not possible directly, however, because 
according to the non-linearities involved there have to be formed 
non-linear functions of the real parts. On the premises made 
above, that only products or powers are allowed (which are the 
non-linearities occurring most frequently in practical control 
systems), this difficulty can be circumvented. It can be shown 
easily, that for a product of real parts the following formula 
holds: 

0tx x 0tx 2 — ~0t (x±x 2 H-x^x*) (10) 

For powers of real parts one gets 


{91 xf 


2 k-x 

1 

\k- 1' 


- fe -1 

E ( * lx' 1-1 ’**” 
_v = 0 V v 


for k odd 


i E ( k ^* k ~ v 

Lv=o V v 


X x +- 


2 \k/2 


5?x* J 


( 11 ) 


for k even. 

Considering the formulae (10) and (11) and also the trivial 
relation 

M x* = 0tx (12) 


one can manipulate in eqn (9) the complex variables directly 
instead of the real parts. Thus a comparison of coefficients now 
possible yields a set of complex algebraic equations: 


fo %av> ^av) 0 

ft (0eiv Xeu ? ^crv) 0 


f n (^eu 5 X e n , X av9 X av ) 0 


ju — 0... m 
v = 0... n 


(13) 


This set of equations now describes the behaviour of the non¬ 
linear system for an input according to (7). It thus represents the 
describing function in implicit form. It may be written explicitly 
according to the definition (3): 



(14) 


If the method presented above is applied to a linear system, 
all equations of the set (13) would prove to be identical and their 
solution would give the frequency response function. In the case 
of a non-linear system with products of the variables, a linear 
set of complex algebraic equations results. As will be shown in 
the first and second example, the solution of this set with an 
unlimited number of higher harmonics leads to a describing 
function, which has the form of a continued fraction. If the 
given differential equation contains powers of the variables, the 


D 


? E s**""*. 

,«=o 


? E jncox^* E*. 

H =0 v = 0 




E J VC0X a 

V = 0 


^JVCOt 


= 0 


(9) 
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explicit solution of the set (13) proves to become increasingly 
difficult. In a third example it will be demonstrated that an 
explicit solution can be derived only if a limited number of 
higher harmonics of the output is taken into account. 


Examples 

(1) The first simple example has been chosen so that the 
application of the general method can be illustrated in detail. 

Consider a control system component characterized by a 
differential equation 

^~=x £ (t) + x e (t)x a (i) (15) 

The non-linearity encountered is the product (/) x a (t). The 
term ‘non-linearity’ seems to be incorrect, since the differen¬ 
tial equation (15) is linear with a variable coefficient, if (/) 
is a given time function. It is justified to speak of a non-linearity, 
however, if eqn (15) is used to describe the transfer behaviour of 
the system. Due to the product x e x a , the output x a ( t ) depends 
non-linearly on the input (/). 

According to the general method given above to calculate 
the describing function, the expressions (7) and (8) are introduced 
into the differential equation. For simplification the input 
function (/) is assumed to be of the form 

x e (t) = M £ x efl e ill<ot = (x e0 + & el e jat ) (16) 

n = 0 

The output x a ( t ) may be expressed in a series of harmonics 

*«(*)“# t X av QjVOit (17) 

v = 0 

With these terms introduced into the differential equation 
(15), and observing the formula (10) for a product of real parts, 
the following equation results: 


Eqn (18) 


Eqn (18) may be written in the form 


Eqn (19) 


t 


By comparing now the coefficients of the terms with the 
same frequencies, one gets the following set of complex equa¬ 
tions: 


X e0 + * e0 XaO+Y^ *el X fll = 0 


( 20 ) 


X al(jW X eo) ^^el\^4rX a0 4—^-X al 

)=° 


Xa2 (2 jCO - X e0 ) X a3 ) 

=0 


x a3 (3 jCO X e0 ) — ( X a2 — X a4 ) 

=0 

(21) 


x an (njco - x e0 ) ~^f(x a -X e ( „ +1) )=0 

The variables x av of the set (20) and (21) may be eliminated 
successively, giving the describing function 

x el 

1 4~X 0 q 


JCD~X e0 —-X el ■ 


2 jco- 




— ~x 2 
4 Xel 


1 


3 jCO-XeQ 


i.2 1 

4 el 4jco... 

( 22 ) 

In addition to (22), eqn (20) is valid for the constant components 
x a0 and x e0 . 

It proves to be unnecessary in this case to limit the number of 
harmonics considered in the output function (t). For n oo 
an infinite continued fraction results. 

For a practical calculation of the describing function the 
general form of eqn (22) seems to be only of academic interest, 
since the results are quite satisfactory by taking into account 
only the first and second harmonics. If a digital computer is 
used to numerically calculate eqn (22), the consideration of 
higher harmonics is possible without any additional difficulties. 
In this case the describing function can be determined from 
eqn (22) with any degree of exactness. Furthermore, the general 
formula (22) gives insight into the behaviour of the system 
under consideration, showing the effect of higher harmonics on 
the results and making possible an estimate of the errors caused 
by omissions. 

(2) As an important practical case of a component with a 
differential equation of the form discussed above, the nuclear 
reactor may be taken as a second example. In the region near a 
steady state power the kinetic equations 4 are: 


1_ d x a (t) 
P d t 


--X e (t) + x e (t) x a (t) - x a ( t ) + X a iCi ( t) 

i = l 


dCj(t ) 

d t 


= ^[>a(0-Ci(*)] 1 = 1...6 


(23) 


Eqn (18) 


t Eqn (19) 


+*.i»—) +Y®\i i (18, 

Lv — 0 v = o J 

t 0 ^ eiV(Ot ~Y^{i o ^ve ;c “ (v+1)f +i (19) 
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In this equation, /, /?, and are constants. The input 
(0 (the so-called reactivity) will be assumed in the form of 
eqn (16). The periodic output ( t) (the relative deviation of the 

power from a steady-state value) and the variables c t (/) (rela¬ 
tive variation of the concentration of the zth delayed neutron 
group) are expressed as an infinite sum of harmonics: 


00 


*«(0 = # Z x av S JVOt 

(24) 

V = 0 


oo 

C t (f)=& Z c iv e Jvrat 

(25) 

v = 0 


With the abbreviation 


G(j<o) = ja>( + Z ‘ . ) 

VP i=i JW + AiJ 

(26) 


an insertion of these expressions (24) and (25) into the differential 
equation (23) conducted in the same way as in the first example 
yields the set of complex equations 


It is assumed that the describing function of a system has 
to be calculated, characterized by the equation 


m 


Z 


d^CQ 

d t l 


+ x 2 a {t) = x e (t) 


(29) 


Analogous to the preceding example the input variable is 
supposed to consist of a fundamental harmonic and a constant 
term x e0 . 

The periodic variables are expressed in terms of their harmonic 
content: 


*.(*)=« Z 

P = 0 

t ^ Jvat 

v — 0 


(30) 


With these expressions inserted into the differential equations 
(29) and applying the formula (11) which takes the form 


X e o(i+X a0 ) + Y^KlXal 


= 0 


1 _ 


^al[G(j^)-X e0 ]~X ei [ l+XaQ+y X a2 ) =0 


(M x) 2 =—& (x 2 + xx*) 
(27) the following equation can be derived. 


Xavl G U VO >)- X eo]— 2-* e l(x«(v-l) + X 8(v + 1) ) = 0 

v = 2... oo 

Elimination of the variables x av gives the describing function 


(31) 


Z ® Z (;'vffl) I x av e Jfv “ < 

1=1 v=0 


+ - 


\2 


Z Xav Qjv(0t ) +( z *«e / "*)( Z x* y e~ J ' vcot 

j= o J 


v = 0 


,v = 0 


Eqn (28) 


It can easily be seen that the form of the describing function 
depends only on the type of the non-linearity. As in this second 
example, only the linear part of the equations is different from 
that of the first example, the resulting describing function (28) 
proves to be identical to the function (22), if only jvco is replaced 
by G (jvco). 

In Figures 2 and 3 the gain and phase of the describing 
function of the nuclear reactor are shown. In the case chosen 
here (x a0 — 0), both the gain and phase are reduced with 
increasing amplitudes of the input reactivity. Hence the im¬ 
portant conclusion follows, that stability will increase by the 
effects of the non-linearity compared to the linearized case (the 
linearized frequency response is the curve with zero amplitude) 4 . 

(3) In a third example a transfer system with a power of 
the output variable is considered. A practical case of a system 
of this kind would be, for example, the heat transfer from a 
surface to a cooling medium. It can be described by a differential 
equation containing the fourth power of the output variable 4 > 5 . 
For the sake of a simple illustration an investigation is made of 
a similar system with a square of the output variable. 


= <* I x. p e"“* (32) 

fi = Q 

Comparison of the coefficients of the terms of identical 
frequencies yields the following set of algebraic equations: 


1 " 
Z i = 1 


Z 0"^) %al "h X a i + Z 1) 


= *«0 
= *el 


W l f H t 

Z (2 j(0) X a 2 +— ( X al +2 Z Xai%a(i + 2) 

1=1 Z V i =1 


= 0 


Z (FJ^) ^an -^an "h n Z — i) attain + i)) ^ 

1=1 z i=l 


n — i^ 1 (33) 


* Eqn (28) 


iV = C£i = 

Xel 


1 + X a0 


G (jco) - 


v-_ L. _ 

G (2 jco) — x e0 — —St 2 el 


G(3»-x e o-T^__l_ 


(28) 
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Contrary to the linear set of complex equations of the 
preceding examples two major difficulties arise: (a) Since the 
differential equation here was non-linear, the complex algebraic 
equations are now also non-linear, (b) The conjugate complex 
variables are contained explicitly in the resulting equations. 

According to these complications, the analytic solution of the 
set of eqns (33) is possible only if a limited number of higher 
harmonics is allowed. Considering a second harmonic in 
addition to the d.c. component and the fundamental harmonic, 
the following formula can be derived from (33) for the describing 
function: 


N= 




E (jco) l +2x, 
1 = 1 


a 0 


+ - 


1 = 1 


M 2 

(2jco) l + 2x a0 


(34) 


According to the first equation of (33) the d.c. component x a0 
is a function of the input variables x el and x e0 : 



IV 2 

m 

E (2jco) l + 2x a0 

1=1 I 



(35) 


Eqns (34) and (35) can be solved numerically only by trial and 
error. It would thus be more practical to calculate x al and the 
describing function N directly from the set of algebraic equations 
(33). The formulae (34) and (35) are only valuable to the extent 
that they show explicitly the influence of the second harmonic 
which is seen to depend on the square of the amplitude of the 
input. It can be neglected for very small amplitudes. 


Conclusions 

The describing function of a control system component can 
effectively be evaluated in the frequency domain, if the non- 
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linearities involved consist of products or powers of the variables. 
A set of complex algebraic equations can be derived which 
contains the describing function in implicit form. For systems 
with other types of non-linearities than products or powers the 
functions of the real parts introduced could not be solved 
explicitly, which means that the method would not be applicable 
any more. In such cases the describing functions would have to 
be calculated in the time domain 3 ’ 2 . 

In the case of systems which contain only products of the 
input and output variables (which, for example, occur most 
frequently in adaptive control systems), or in other words which 
are characterized by linear differential equations with variable 
coefficients, the resulting complex algebraic equations will be 
linear. They can be solved explicitly for an unlimited number of 
higher harmonics in the output, and the describing function 
resulting has the form of an infinite continued fraction. 

The complex algebraic equations will be non-linear, if a 
system characterized by a non-linear differential equation with 
powers of the variables is considered. Numerically the describing 
function can be calculated by iteration from these. If only a 
fundamental harmonic together with one or very few higher 
harmonics are taken into account, an explicit expression may be 
derived. 
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Nomenclature 

X e (t) 

Input variable of a transfer system 

X a (t) 

Output variable of a transfer system 

X 

Amplitude (maximal value) of a sinusoida 
time function x ( t ) 

x = x e/0 

Complex amplitude coefficient 

? = jce- 

■j<t> Conjugate complex amplitude coefficient 

0) 

Frequency in rad/sec 

N 

Describing function 

& 

Real part 

Index 0 

D. C. component 

Index 1 

1st harmonic (fundamental) 

Index 2 

etc. 

2nd harmonic 


DISCUSSION 


W. W. Seifert, Massachusetts Institute of Technology , Cambridge 39, 

Mass., U.S.A. 

The author is to be congratulated for a very lucid presentation of an 
interesting new method for deriving the describing function for systems 
in which the non-linearity appears in the form of products or powers 
of the variables. The method discussed carries out the evaluation 
completely in the frequency domain and offers the advantage of ena¬ 
bling one to consider a system input which consists of a number of 
harmonics rather than a simple sinusoid, and simultaneously permit¬ 
ting computation of the describing function when the output variable 
is described as a harmonic series. The real advantage of carrying out 
this computation in the frequency domain is that the whole process is 
reduced to an algebraic one, which, while the resultant equations may 
be very difficult to solve, is at least, straightforward to formulate. The 
difficulty of solution, however, follows directly from the basic com¬ 
plexity of the problem and certainly is no more severe in the proposed 
method than if the more usual time-domain approach were used. 

On the other hand, for this method to be applicable, the non¬ 
linearity must be expressible in terms of products or powers of the 
variables. This is a relatively severe restriction but, in a sense, it is 
fortunate that the method applies to this type of system rather than 
some of those handled readily by time-domain methods. Systems in¬ 
volving products frequently lead to frequency-dependent describing 
functions whose derivation requires more ingenuity than those de¬ 
pendent on amplitude only. While the amount of actual labour involv¬ 
ed is not reduced by Lauber’s method, the process is at least well 
defined. Unfortunately, solution of the resultant equations may be too 
difficult to handle except by machine methods, with the result that in 
these cases the real advantage of the describing function method, 
namely that of being a simple to use engineering tool, is lost. 

The method appears to offer no advantage whatsoever when the 


output of the non-linearity is related to its input as a simple power, 
such as a square or a cube function. Here, it appears simpler to divide 
the problem into the linear portion and the non-linear portion for 
which the derivation of the describing function is essentially trivial, 
and handle the combination by the usual describing function method, 
than to develop one describing function for the overall system by 
Lauber’s technique. 

One problem which remains unanswered with either approach is 
that of determining the error which results when the output of the 
non-linearity is described by a few terms of a series, as is required if 
the amount of effort in obtaining numerical results is to be kept with¬ 
in bounds. 


R. Lauber, in reply 

I thank Professor Seifert for his very profound comments, but regret 
that I cannot agree with the statement that the amount of labour is 
not reduced by my method. Making the calculation in the frequency 
domain, the variables are vectors containing both amplitude and phase. 
When the calculations are made in the time domain, amplitude and 
phase are treated separately so that the number of variables is doubled. 
This reduction of the number of variables by a factor of two might be 
very important in calculating describing functions of complicated 
systems, especially if higher harmonics are considered. T think that 
this is one of the advantages of my method. 

I do agree with Professor Seifert that in those cases where the non¬ 
linear part of the system can easily be separated from the linear one, 
this should be done. The method presented is aimed at those problems 
where such a separation cannot easily be carried out. 

To the question of the errors which result when considering only 
a few harmonics, no general answer can be given. I can only mention 
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that in the case of the nuclear reactor (the second example in my 
paper), an investigation to determine these errors has been carried out. 
It showed that a calculation without considering any higher harmonics 
in the output function leads to a small error in amplitude, but to quite 
an essential error in the phase of the describing function. A calculation 
without consideration of any higher harmonics leads to an error in 
phase A <£ N = 10° at the frequency co - 0*1 S" 1 (. Figure 3 of the 
paper). If only the second harmonic is considered, the error at this 
point may be reduced to A N = 1 °. So in this example the con¬ 
sideration of one harmonic in addition to the fundamental harmonic 
leads to quite satisfactory results. 


W. Filipczak, Institute of Nuclear Research, Swierk p. Otwock 
Warsaw, Poland 

In your example concerning the nuclear reactor you reached some 
conclusions which are not clear to me. In Figure 2 of your paper you 
found that the absolute value of the describing function |IVj is de¬ 
creasing with the input amplitude X ei (the reactivity of the reactor). 
This does not agree with my view. I think the higher the disturbance 
m the reactivity the higher should be the value of | A|. 

As I am an application engineer in the held of nuclear reactor 
control, I am interested to know the advantages of the describing 
function method over the usual simpler methods of analogue computer 
investigation. Is the application of the describing function method in 
this field only of purely academic interest? It may be that I omitted 
some important factors, but in the field of nuclear reactor control I 
do not see the urgent need of the describing function method. At the 
stage of its present development I am not sure that it can be easily 
and fruitfully applied to nuclear reactor control problems. 


R. Lauber, in reply 

I thank Mr. Filipczak for his questions which are very important to 
an engineer working on practical problems in the field of nuclear 
reactor control. The answer to the question, if the amplitude of the 
describing function is always decreasing with increasing amplitudes 
of perturbation, is different for different examples. The important 
factor in this respect is the d.c. component X ao at the output function 
Xa(t), and this d.c. component Xao in turn depends on the feedback 
of the reactor. 

Figure 2 of my paper is valid for X ao = 0. In the cases of pressur¬ 
ized water and boiling water reactors, the factor X m will not be zero, 
but so small, that still the amplitude of the describing function is de¬ 
creasing with increasing amplitudes of perturbation. More information 
on this question can be found in Reference 4 of my paper. 

The second question from Mr. Filipczak on the usefulness of the 
describing function method to reactor control problems cannot be 
answered in a general manner. At any rate, analogue computer meth¬ 
ods are always necessary if transient responses have to be calculated, 
since the describing function method is not valid for the investigation 
of transients; but even for stability investigations it might be easier 
to use an analogue computer if special examples of control systems 
have to be considered. 

On the other hand, the analogue computer study does not give 
general information, and if such information is wanted, an analytical 
method is necessary. A very instructive example can be found in 
Reference 4 of my paper. There it is concluded that for the case of 
boiling water reactors a linear treatment of the kinetic equations is 
always justified since stability will increase in the non-linear case with 
increasing amplitudes of perturbations. Such a general statement could 
never be made by interpreting diagrams computed with the aid of an 
analogue computer. 
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The Describing Function Method Applied for the Investigation 

of Parametric Excited Oscillations 

A. LEONHARD 


Summary 

In control systems analysis there occur, occasionally, oscillatory loops 
including elements with multiplicative interaction. The investigation 
of such systems, using the describing function method in the usual 
manner, i.e. by neglecting the harmonics, may produce entirely wrong 
results. If the multiplier is excited by oscillations of various combina¬ 
tion frequencies, it is quite possible that one harmonic may interact 
with a fundamental frequency component to form a new not-negligible 
component of fundamental frequency. In the simplest case a harmonic 
component y 2 = a 2 sin 2 cot, when multiplied with a component of 
fundamental frequency y x = a x sin cot will produce an output of the 
form - 

y s = cos wt — -h a x a 2 cos 3 cot 

the fundamental component of which is a function of the harmonic 
excitation y 2 . The harmonic components thus influence, often to an 
important degree, the fundamental component and the stability. This 
effect of interaction of various frequencies in the non-linear element is, 
of course, the basis for parametric oscillations. 

When applying the describing function method to systems of this 
type it is shown that this is advantageous—the harmonics must be 
taken into account in as much as they influence the fundamental 
frequency oscillation. Theoretically there is an infinite number of 
these harmonics. It is shown that satisfactory results can be obtained 
by including only two or three prevalent harmonics. The advantage of 
this method, as opposed to purely mathematical treatment, is the fact 
that it can be used with systems of any order and that it is a natural 
extension of the well-known linear control theory. 

Sommaire 

Dans certains systemes de commande, il existe des boucles instables 
comprenant des dldments a interactions multiplicatives. L’analyse de 
tels systemes par la mdthode de la frequence fondamentale conduit 
quelquefois k des rdsultats erronds. Si le multiplieur est excite par une 
combinaison de frequences, il est tout a fait possible que l’une des 
harmoniques agisse sur l’une des frequences de base pour produire 
une composante non ndgligeable de cette derniere. Dans le cas le plus 
simple, la multiplication d’une harmonique 

y 2 = a 1 sin 2 (ot par une fondamentale 

y i = s i n cot produit une sortie de la forme: 

y s — a i a 2 cos tot — i a i a 2 cos 3 cot 
dont la composante fondamentale est une fonction de Tharmonique 
excitatrice y 2 . Les harmoniques influencent ainsi, dans une large 
mesure, la composante fondamentale et la stability Cet effet 
d’interaction de frequences differentes dans l’dlement non-lindaire est 
certainement k la base d’oscillations parametriques. 

Dans ce rapport, on presente une extension de la mdlhode de la 
frequence fondamentale en tenant compte des harmoniques. Theori- 
quement, le nombre de ces dernidres est infini. On montre que dans 
la pratique, il suffit de tenir compte de deux ou trois predominantes. 
Une telle approximation peut etre avantageusement utilises avec des 
systemes de n’importe quel ordre. Elle constitue une extension naturelle 
de la theorie lineaire. 

Zusammenfassung 

In der Regelungstechnik kommen schwingungsfahige Gebilde mit 
Multiplikationsstellen vor. Untersucht man solche Kreise mit Hilfe 


der Beschreibungsfunktion in ihrer herkommlichen Form mit Ver- 
nachlassigung der Oberwellen, so kommt man unter Umstanden zu 
vollkommen falschen Ergebnissen. An den Multiplikationsstellen 
treten verschiedene Frequenzkombinationen auf, und es kann durch- 
aus der Fall auftreten, daB eine Oberwelle etwa zusammen mit der 
..Grundwelle eine mit der Grundwelle gleichfrequente Schwingung als 
AusgangsgroBe liefert. Der einfachste Fall liegt vor, wenn die Ober- 
schwingung y — a sin loot multipliziert wird mit der Grundschwingung 
b sin cot, also a sin loot, b sin cot = 1 l 2 ab cos cot — x l 2 ab cos 3 cot. Am 
Ausgang erscheint also eine von der Oberschwingung abhangige, der 
Grundschwingung gleichfrequente Schwingung. Die Oberschwingung 
beeinfluBt also hier, unter Umstanden sogar in erheblichem Aus- 
maB, die Grundschwingung und damit die Selbsterregung. Para- 
metrische Schwingungen, mit denen sich dieser Vortrag vor allem 
beschaftigen soil, kommen uberhaupt erst dutch solche Frequenz- 
Kombinationen zustande. 

Will man also auch bei solchen Kreisen mit der Beschreibungs¬ 
funktion arbeiten und zeigt es sich, daB man dies mit Vorteil machen 
kann, so miissen die Oberwellen insoweit berucksichtigt werden, als sie 
die Grundwelle beeinflussen. Dies sind theoretisch im allgemeinen 
unendlich viele. Es laBt sich aber zeigen, daB schon bei einer Be- 
schrankung auf nur zwei Oder drei Oberwellen vielfach brauchbare 
Ergebnisse erzielt werden konnen. Die Vorteile dieser Behandlungs- 
weise gegeniiber anderen mathematischen Verfahren liegen einmal 
darin, daB die Zahl der Freiheitsgrade des Systems keine Rolle spielt, 
und zum anderen, daB sich die Methode an die bekannte lineare 
Regelungstheorie anlehnt. 


Introduction 

The describing function method, sometimes called the harmonic 
balance concept, is already well known and provides a powerful 
means for the investigation of self-excited oscillations of electric 
machinery or for the stability analysis of non-linear control 
loops. 

The basic assumption for the describing function concept is 
that the fundamental harmonic of an oscillation in a closed 
feedback loop is not influenced by the higher harmonics, which 
are in general damped by the lags of the loop anyway. All the 
higher harmonics will be neglected. Experience has taught that 
the method is a very powerful tool as long as the basic assump¬ 
tion is true, that the fundamental harmonic is predominant and 
not affected by the higher harmonics. 

But there are technical systems where a multiplication of 
two variables is included in the dynamics 1 ' 3 . Multiplication of 
variables in a dynamic system occurs frequently in all systems 
with time-varying parameters. Certain non-linear problems can 
be approached in the same way. 

If one investigates such feedback loops with the aid of the 
usual describing function concept, in its conventional way, 
neglecting all the higher harmonics, one may obtain under 
certain conditions grossly incorrect results. 
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yields an output <x 3 which gives at the output of B, after being 
multiplied by y (t), two oscillations with the frequencies v 
and 5 v designated by the symbols (3 lb and f} 5 . The dropping of 
the fifth harmonic /? 5 is usually allowed. 

The two oscillations ft lb and /3 la are added and one obtains 

fila~ a l“®ll filb == ^3 ' ^31 

a 3 =^F 3 + 

The condition for the stability boundary is easily determined 

^^=£ 11 F 1 + j B 1 3F 3 B3 1 F 1 = 1 (20) 


F 



Figure 3. Corresponding block diagram, to Figure 2 considering the 
third harmonic 


third harmonic) y = 1-58. But one obtains for the stability 
analysis with the aid of eqn (23) (with consideration of the third 
harmonic oscillation) no real value for y, which means there is 
no oscillation with the frequency v = 0*5 possible for fi = 1. 
As will be seen later, the fundamental harmonic for pi = 1 is no 
longer v — pt/2, but v = pi. 

First, a second-order differential equation, like eqn (1), is 
investigated, but now it is assumed that the damping will vary 
harmonically. Eqn (24), according to eqn (16), is now 

y" + Co/ + y = y'y sin (jut + \jj) (24) 

(£o is the average value of the damping factor). Provided one 
again assumes steady-state sinusoidal oscillations, (the symbols 
are the same as in Figure 2) oc^ is equal to do^/dr-t and one 
obtains, corresponding to eqn (11) and eqn (9), 

= =a 1 y?(cos^-f ;sini/0 (25) 

or 

/? 3 =a 1 B 13 =a 1 y 7 e # (26a) 

Pib = <x 3 B 31 =cc 3 (-Yye~ J ' 1 '') (26b) 


According to the relation of eqn (9), the following equations 
are easily proved to be correct: 


B n = yl’O'cosi/' —sin f) 
according to eqn (18). 

^13^31 = 4-y 2 

The analogue eqn (17) for jli = 2 yields 

1 


fA = 


j,£ + j,£ e+l 


So the final equations are obtained: 


( 21 ) 


( 22 ) 


If one has first assumed the closed feedback loop to be open, 
the relation for self-excitation is again = 1-0. Following 
this concept one finds the conditions for the determination 
of yj and y of eqn (24). 

If the coefficient of y" is varying in the differential equation 
[it is assumed to be 1*0 in eqn (1)] periodically, one obtains, 
according to the eqns (25) and (26) 


Pia — a i^n — a i 


—y Hi cos ^ 


- sin if/) 


fi 3 — a 1 B 13 _ a x ( + j — y e 7 '^ 


Pib= :( x 3 B 31 =cc 3 




(27) 
(28 a) 
(28 b) 


fila F P 


lb 




j cos \jj — sin xj/ 


, l L 




+ T 2 


, F 


+j- 2 i+i 


.3 pi 


+4 f+i 


-1(23) 


If one performs the stability analysis with the more accurate 
eqn (23), compared with eqn (19), one obtains for y with the 
specific values pi = 2 and 4 and f = 0*5 practically the same 
values as one did with eqn (19). The influence of the third 
harmonic can therefore be neglected for these two problems. 
(That means for pi ^ 2, the frequency of the sinusoidal para¬ 
meter variation must be equal or larger than twice the resonance 
frequency.) One determines for the value pi = 1, which means 
for v — pi = 0*5, with eqn (19), (without consideration of the 


For the frequency ratio the assumption v/pi = 0*5 is made. 
With this the following oscillations with the frequencies besides 

> 1 = °‘ 5 P wili occur ^3 = 1 1 *5 F ; ^5 = 2*5 pt ; = 3■’5 pi ; v 9 = 4*5 ii 
and so on. 

Which of these oscillations will be the most important de¬ 
pends on the resonance frequency. It should be noted here that 
the stability analysis can be performed with each of these 
oscillations. 

The ratio of the fundamental harmonic to the varying para¬ 
meter frequency of v/pi = -J-, with the corresponding higher 
harmonics, is most likely to occur, but it is not the only possible 
one, Considering Figure 4, one will find, that a frequency ratio 
with the values v/p - 1 is possible too, with the frequencies for 
the higher harmonics ^ = 1*0 pt; v 0 = 0; v 2 = 2 pi; v* = 3 u 
and so on. 

The stability analysis for those problems is performed 
exactly in the same way as before with the value v/pi = 0*5. 
All that remains to be done is to choose the proper values for 
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Figure 4. Block diagram for the calculation of the stability boundary 
v/fi = 1-0 

the transfer functions F and B. According to Figure 4 one 
obtains for instance 

Pi ~ ^12 a l 
or 

/5l6 == ^21 a 2 

and so on. 

For an arbitrary system A , a self-excited oscillation can 
sustain either with one or other frequency ratio dependent on 
the frequency ju and from the fixed parameters of the system A 
Figures 5 and 6 show the corresponding results which have been 
computed with this method. 


and 6. The deviation compared with the calculated values is 
negligible. 514 

In Figure 6 the value y dependent on the frequency of the 
varying parameter at the stability boundary has been plotted 
for a fixed parameter system A with the resonance frequency 
kept constant equal to one. The plot of Figure 6 demonstrates, 
that for f = 0*5, and for zero-damping, the measured and the 
calculated values are comparable for frequencies larger than 
fi!co e = 0*75. In Figure 7 the analogue computer plots y (r) and 
y (t) for ix = 1*5 (v = 0*5 ju) and ju = 0*75 (y = fx) are shown. 



12 3 4 



Figure 6. Amplitude y of the parameter oscillation at the stability 


1 

I 


Figure 5. The amplitude of the parameter oscillation y <jx = con¬ 
stant = 1) at the stability boundary , dependent on the natural frequency 
of a second-order system 



boundary of a second-order system with the natural frequency l as 
function of the frequency of the varying parameter 



In Figure 5 the value y at the stability boundary has been 
plotted, dependent on the resonance frequency co e = ]/K for 
zero damping and for f = 0*5 with the value [x — 1 kept 
constant, y is the amplitude of the parametric exciting oscillation. 
The meaning co e = ]/K can be seen from eqn (3). 

The plots for f = 0 are known 4 as Ince-Strutt stability dia¬ 
gram. These diagrams can be computed easily with this method 
with the aid of the transfer function and the describing function 
method shown in this paper, with the consideration of the first 
and third harmonic for the value vfrx — 0*5 and with the con¬ 
sideration of the d.c. component, the first and the second 
harmonic for the ratio vj/x = TO. The calculations for the 
damping factor £ = 0*5 are only slightly more complicated. 
It has been proved with the analogue computer that the influence 
of the higher harmonics is small and mostly negligible. The 
results found by analogue simulation are dotted in Figures 5 



Figure 7. Undamped oscillations of a parametric excited system: 
(a) p = T5, v — 0*5 fx; (6) [x = 0*75, v = p 

* Acknowledgement is due to Dr. Gerhard Schweizer, who per¬ 
formed all the analogue computations. 


25 







A. LEONHARD 


One finds that for p — 1*5, the third harmonic which has been 
considered in the calculation, and for p = 0*75, the d.c. com¬ 
ponent and the second harmonic which have been taken into 
account, are predominant. In Figure 8 the plots for p ~ 2 
(v = 0-5 [£) and for p — 1 (v = p) are represented, y (r) is 
almost sinusoidal. A d.c. component will occur for p — 1. 

The method cannot be applied successfully for the range 
of F from 0T to 0*6. Figure 6 does not give reliable comparisons 



v = 1 



Figure 8 . Undamped oscillations of a parametric excited system: 
(a) p = 2, v = 0*5 p; (b) p = 1, v = 1 p 



Figure 9. Undamped oscillations of a parametric excited system: 
p = 0*2, v = 1 ju 

between the measured and calculated values. The reason is that 
many more harmonics should be considered. Figure 9 shows 
the plot for y (r) and y (r) for the value p = 0*2. The parameter 
variation is so slow that the transients within one period will 
almost die out. During the next half period y (r) will rise about 
aperiodically. The final values of the second half period are the 
initial conditions for the third half period and so on. One 


possible approach to the problem would be the replacement of 
the sinusoidal oscillation of the variable parameter by a square 
wave. 

Linear System with Two Periodically Varying Parameters 

The analysis for time-varying systems has been limited up 
to now to the investigation of problems with one variable 
parameter. However, there are systems with two varying para¬ 
meters. It is now shown with the aid of a specific problem—the 
oscillation of a swing—how this method may be applied. 

It is well known that the amplitude of the oscillation of a 
swing is kept constant or excited by elevating or lowering the 
weight. This energy overcomes the unavoidable friction. 
Limiting oneself on the stability analysis, where the swing is 
oscillating with constant amplitude and the overall damping 
due to the friction of the bearings is kept zero by the energy 
of the variable parameters. 

The oscillation of the swing can be represented by the 
differential equation where the length of the pendulum is 
varying 5 

cc"mr 2 + a' (p + 2 rr'm) + mgr sin a = 0 (29) 

m stands for the mass, r is the length of the pendulum, a the 
angle of the pendulum, g is the gravity acceleration, p the 
damping (assumed to be proportional to the velocity). 

The length of the pendulum r is varied by elevating and 
lowering the centre of gravity. A sinusoidal variation is assumed 
and one obtains 

r = r 0 - Ar sin (co 0 1 + \j/) = r 0 [1 - y sin (cd 0 1 + ^)] (30) 

r'= — r Q ycx) 0 cos (co 0 t -f \jj) (31) 

Eqn (29) can be written in the following form (oc is assumed to 
be small, which means sin oc & oc, and T 2 = r 0 /g is substituted) 

a" T 2 + a' £ T + a = a" T 2 y sin (co 0 1 + ij/) - Ty sin (© 0 / +1/0 

+ a' T 2 2 co 0 y cos (co 0 t+ip) (32) 

f is the normalized damping: 

— p ~— 

™ r o (gro)* 

In the last equation, use was made of the following relation 

T-ysin{ mo t+Tf 1+VSin(OJ ° t+f) 

for small values of y. 

Introducing the normalized time t = t/T and ojT = Q, 
co 0 T — Q,j one obtains 

a" -f cc'c + a = a."y sin (I2 0 r + \J/) + a.’2Q 0 y cos (Q 0 r + \p) 

sin (OqT + i/O (32 a) 

One knows from experience that one has to raise the weight 
at the lowest point of the swing oscillation, that means at a = 0, 
if one intends to excite the swing oscillation. During one period 
of the oscillation one has therefore to raise and to lower its 
weight twice. The frequency of the parametric oscillation is 
therefore twice the frequency of the swing oscillation. The 
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easiest way, but not the only one, to keep the swing going, is to 
oscillate with the resonance frequency. Therefore, putting 


co 0 — 2 co e «“ 

(33) 

Q 0 = o)qT = 2 

(34) 


According to eqn (32) 1/T is the resonance frequency of the 
swing without consideration of the friction. £ is always small 
compared with 1. The same is true for y, therefore the variable 
oc'gy sin (Q 0 r + ip) in eqn (32b) can be dropped. 

At the boundary of the stability the amplitude of the oscilla¬ 
tion is constant. Considering the fundamental harmonic and 
the third harmonic, one can write with r = t/T: 


<x 1 = 0 L i sim; 1 

a 3 =a 3 sin(3T + <p) j 

(35) 

Then the following equations are obtained 


—0t x COST | 

a 3 = 3a 3 cos(3 x + (p) J 

(36) 

ol[ = — sin t | 

a 3 = -9a 3 sin (3 T + <p)J 

(37) 


The aim now is to calculate the value y = Ar/r 0 so that an 
undamped oscillation can sustain. 

Considering the fundamental harmonic and the third 
harmonic, one can perform the calculation with the aid of the 
block diagram of Figure 10. Part A of Figure 10 corresponds to 
the fixed parameter system, which is equivalent to the left side 
of eqn (32 b). The blocks B and C are additional systems, which 
stand for the right side of eqn (32b), where both differentials 
are multiplied by the parametric oscillation. The feedback 
loop is now assumed to be open at the designated spot, but only 
for the fundamental harmonic. It is closed for the third harmonic. 

According to Figure 10, the following relations can be 
obtained 

=a 1 jB 1 x ; ^Bl = a 3^3 U ^ci =a 3^31 | 

Pb3 = oqI3 13 ; Pc3 = a iCi3 I 

a 1 =8iF 1 ; a 3 = (P B 3 + Pc3)F3 ' 


For the stability boundary one gets 

1 __ bi +ftci + + ^C1 

£ 1 

=F X [jB 1 i + C 11 + (Bi 3 + C 13 ) F 3 (B 31 + C 31 )] (39) 

=Fi [B n + C n +F 3 (B 13 B 31 + C 13 C 31 +B 13 C 31 + C 13 B 31 )] 

Substituting Q = 0-5 Q 0 = 1 the relations yield 

1 _ 1 
F '-(jQ ) 2 + jl;Q + rji 

p 3 =_ 1 - 

3 -9+j3£+l 

B u =y-4y (jrcosi/'-sini/O 

According to eqn (18) and eqn (9), considering 2 Q 0 = 4, one 
obtains 

C u = -yy O'cost/'-sin \j/) 

With the aid of eqns (9) and (32) the relations can be determined 

B 13 B 31 =-12y 2 

Ci 3 c 3 i= -y? 2 j 


£'i3®3i — 3'y 


Bi 3 C 31 — 9y 

Finally one will find 

(40) 

Neglecting for the moment the third harmonic, the part with 
y 2 drops. In this case one obtains the two equations sin ip = 0 
and | = 1-5 y cos With the value £ = Ol one calculates 
e.g. y = 0-1/1-5 = 0-066. With £ = 0-1, which corresponds to 
a damping time constant of the swing oscillation of 


‘-3 


2-25 


y(-l-5sin^ + jl*5cosi/0-y _8 + /3 gj 



Figure 10. Block diagram for the determination of the stability boundary 
of a swing oscillation with two periodically varying parameters 


o T 20 

T -_=20 T=— T =3 T 

the ratio of the amplitudes after three periods of the oscillation 
is 0*36. In this case y must be y =0-07. From this it follows that 
the length of the pendulum has to be oscillated ± 7 per cent, 
so that an undamped oscillation can be sustained. If the third 
harmonic is considered, then the following two equations are 
obtained: y * 1*5 sin ip — y 2 * 0*28 = 0 and 0*1 — y * 1*5 cos ip — 
- y 2 ■ 0*01 = 0. 

Nearly the same value for y is obtained from these two 
equations as before, without considering the third harmonic. 
It is assumed in Figure 10 that two parametric oscillations of the 
variable parameters are either in phase or 90° out of phase. 

No difficulties arise if one allows an arbitrary phase angle 
between the two oscillations. In this case one has to introduce 
the fixed phase angle between the two parametric oscillations 
besides the phase angle ip of the oscillation of the system and 
one of the parametric oscillations. 
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Even the two frequencies of the parametric oscillations must 
not be the same. But system oscillations will only occur, due to 
both variable parameters at certain ratios of the frequencies. 
The ratio must be such that each parameter oscillation, modu¬ 
lated by the system oscillation, yields one component of one 
frequency of the system oscillation. For a frequency ratio of 1:2 
a system oscillation of the lower frequency can occur. Figures 3 
and 4 can be used as before for the analysis. 


Arbitrary Linear System and Non-harmonic Parametric Oscillation 

. ^ , can . easiJ y be seen from the stability analysis, so far, that 

investigations can be made for any linear system, which must 
not be of the second order, with periodically varying para¬ 
meters. The linear part with fixed parameters is represented by 
the transfer function F(jco) in any case. The describing function 
remains the same. 


If one parameter varies periodically but not harmonically, 
the stability analysis of each oscillation, determined by the 
Fourier series, can be investigated separately. In general the 
consideration of the fundamental harmonic will be sufficient. 
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DISCUSSION 


P. DoRato, Polytechnic Institute of Brooklyn, 333, Jay Street, Brook- 
lyn I, N.J ., U.S.A. 

I would like to point out some results obtained by my colleague 
Professor Bongiorno on the stability of periodically-varying linear 
systems presented at the 1962 I.A.C.C. Conference in New York A 
typical result is as follows. 



k(t) 


G(s) 




p 




Figure A 

The system shown in Figure A is stable for any periodically-varying 
gam k (t) if the condition J y s 


max|G(jco)|g 


1 


/v max 

where I k (t) | g k max . The structure of this figure is quite general since 
any time-varying parameter can, as shown, always be isolated as a 
gain. In addition, Professor Bongiorno has obtained estimates of the 
decay time when the system is stable. 

A. Leonhard, in reply 

The case discussed by Professor Dorato-when the gain of the feed- 
back loop k (0 is time-varying—is a special case, which can obviously 

Up at r d by th V m ® tbod exposed in m y P a P er - The criterion cited by 
the Professor, which has been derived using methods differing from 
mine appears, however, as a sufficient but not absolutely necessary 
condition for stability. y 
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On the Inverse Describing Function Problem 

J. E. GIBSON and E. S. diTADA 


Summary 

With the exception of a passing reference in an early note by Zadeh 1 , 
little or no attention appears to have been paid to the inverse describ¬ 
ing function (IDF) problem. In essence this problem is concerned with 
extracting a description of a non-linear element given only its describ¬ 
ing function. The problem is important as the final link in the non¬ 
linear synthesis problem for automatic control systems, for various 
experimental determinations, and finally for itself. 

In this paper an analytic solution to the problem which results in 
expressions in the form of Volterra integral equations is provided. The 
conditions under which the representation for the non-linear element is 
unique are discussed and additional requirements for the unique defi¬ 
nition of non-single-valued non-linear elements are given. As might be 
anticipated the describing function of a memory-type non-linear ele¬ 
ment is not sufficient to obtain a unique representation for the element. 

Practical computational techniques are mentioned for the machine 
solution of the IDF and several examples are given. 

Sommaire 

A part une allusion figurant dans un travail de Zadeh 1 , peu d’et udes 
ont ete consacrees au probleme de la transformation inverse de la 
fonction descriptive (inverse descriptive function (IDF)). Ce probleme 
consiste a determiner la fonction originale d’un element non lineaire 
caracterise par sa fonction descriptive; il se pose lorsqu’il s’agit de 
faire la synthese de systemes de reglage non lkteaires dont les caracte- 
ristiques ont dtd dSterminees experimentalement; d’autre part, il pre¬ 
sente un interet pour lui-meme. 

Ce rapport donne une solution analytique de ce probleme resultant 
de F application des liquations integrates de Volterra. Les conditions 
permettant de representer un dtement non lineaire sont indiquees et 
certains cas particuliers sont traites en detail avec indication des 
methodes d’analyse & utiliser pour traiter ce probleme au moyen de 
calculateurs numdriques. Plusieurs exemples d’application sont donnes. 

Zusammenfassung 

Mit Ausnahme einer beilaufigen Bemerkung in einer friihen Veroffent- 
lichung von Zadeh 1 wurde bisher dem Problem der inversen Beschrei- 
bungsfunktion unseres Wissens wenig oder keine Aufmerksamkeit ge- 
schenkt. Im wesentlichen betrifft dieses Problem die Ableitung der Be- 
schreibung eines nichtlinearen Gliedes, wenn nur dessen Beschrei- 
bungsfunktion angegeben ist. Das Problem ist (als letzter Schritt) bei 
der Synthese nichtlinearer Regelungen fur die verschiedenen experimen- 
tellen Untersuchungen und auch fur sich allein von Interesse. 

Der Aufsatz enthalt eine analytische Losung des Problems, die auf 
Ausdrucke in Form von Volterraschen Integralgleichungen fiihrt. Die 
Bedingungen, unter denen die Darstellung des nichtlinearen Gliedes 
eindeutig ist und die zusatzlichen Erfordernisse fur eine eindeutige 
Definition eines nichtlinearen Gliedes mit mehrdeutiger Kennlinie 
werden angegeben. Wie zu erwarten, reicht die Beschreibungsfunktion 
eines nichtlinearen Gliedes mit Speicherwirkung nicht aus, um zu einer 
eindeutigen Darstellung zu gelangen. 

Rechenverfahren zur maschinellen Losung der inversen Beschrei¬ 
bungsfunktion werden erlautert und einige Beispiele angegeben. 


and the general usefulness of the method in engineering prob¬ 
lems. 

However, in the past, the describing function technique has 
been useful only in analysis. More exactly, it is a powerful tool 
for the investigation of the possible existence of limit cycles 
and their approximate amplitudes and frequencies. 

Recently, synthesis techniques have begun to be developed for 
non-linear systems (Haussler 2 ), in which the goal is to find the 
describing function of the element being synthesized. Therefore, 
for Haussler’s method to be useful, a way must be found to 
reconstruct the non-linearity from its describing function. This 
is called the inverse describing function problem and is essen¬ 
tially a synthesis problem. 

It is not the only case in which the IDF can be useful. 
Sometimes, in order to find the input-output characteristic of a 
physical non-linear element, a harmonic test can be easier to 
perform rather than a static one (which also may be insufficient). 
Finally, it is presumed that a solution to the IDF problem, as 
well as being interesting for its own sake, will add to the under¬ 
standing of the conventional describing function. 

In this paper an analytic solution to the IDF problem is 
presented and its uniqueness under various conditions discussed. 
A number of examples of practical calculations are included. 

Definitions 

Consider a non-linear element and define the input and 
output variables as x and y respectively. The non-linear element 
will be defined only for a sinusoidal input 

x = E sin a>£ = E sin a (1) 

In general, the non-linear element might be allowed to be a 
double-valued function of x, x and E 

y=F(x,x,E ) (2) 

as, for example, would be necessary in a realistic, macroscopic 
model of magnetic saturation. In this paper the element will be 
considered to be independent of frequency, i.e. y = F(x,E). 
If the output y satisfies the Dirichlet conditions, it can be ex¬ 
panded in a Fourier series 

y if)— H- A x (E) cos a 4- A 2 (E) cos 2 a -F ... 

+ A n (E)cos not + ...+B 1 (E)sma + B 2 (E)sin2(x+ ... 

+ E n (E)sin7ia-+... (3) 

where A n (E) and B n (E) are the Fourier coefficients given by 
the following expressions 

1 C 2n 

A n (E)= — / (E sin a) cos n ot doc 

n Jo 

1 f 271 

B n (E)= — / (E sin a) sin n a da 
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Introduction 

The describing function technique has reached great popularity, 
principally because of the relative ease of computation involved 
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The ratio of the /zth harmonic amplitude to the amplitude of the 
input is defined as the «th describing function, a complex 
quantity in general, 

K^ n = g tt (E) + jb n (E)=^l+j^l (5) 


g(E) = 
b(E)~- 


7lE t 

-2 
7 lE 


tc/2 

0 

V/2 


Q (E cos 6 , E) cos0d0 
P (E cos 6 , E) sin 0 d0 


( 10 ) 


The conventional describing function results when n = 1, and 
the subscript is usually omitted. 


Integral Representation of the Describing Function 

With y = F(x,E) as dealt with above, let F 1 (x > E), a 
single-valued though possibly discontinuous function, represent 
F(x, E ) when x is negative and E 2 te E) when x is positive. For 
later convenience let <% = /? -f n/2 t then 

Eqn(6) * 

Let 

F 1 ( x,E ) =P X (x, £) + Q t (x, E) 

F 2 (x, E) =P 2 (x, E) + Q 2 ( x , E) ( ' 7) 


where P x (x, E) = P x (- x, £) and P 2 (x, £) = P 2 (-x, E) and 
Qi (x, E) = -Q 1 ( x, P) and Q 2 (x, P) = - Q 2 (- x, P). Now 
the P ’s and g’s may be written in terms of either F 1 or P 2 with po¬ 
sitive and negative arguments. The linear independence^ the re¬ 
sulting sets of equations is easily checked. Now substitute (7) into 
(6), then let = y + n and after some elementary trigonometric 
transformations. 


where 


Eqn (8) 


t 


P(x,E) = P t (x, E) - P 2 (x, E) 


Q (x, E) = gi (x, E) + Q 2 (x, E) 


(9) 


Since Pj and P 2 are both even functions of x, P is also an even 
function of x. For similar reasons Q is an odd function of x. 

Now let 6 = n - y and employ the odd and even properties 
to obtain 


Now let E cos 6 


= *, then 



b(E)=- 


nE 


0 (E~-x 2 f 
2 [ P(x,E )dx 


(ID 


which are both in the form of Yolterra integral equations. 

The conventional method of computing the describing 
function of a non-linear element requires the knowledge of the 
actual shape of the output signal of the non-linear element 
when its input is driven by a sinusoidal wave. Then a Fourier 
analysis must be performed in order to find the amplitude of the 
first harmonic. This procedure is sometimes rather tedious, 
especially in the case in which the characteristic of the non-linear 
element is not known by an analytic expression but rather is 
given by experimental data. However, by using eqns (11) it is 
not necessary to compute the shape of the output, but only the 
two functions Q (x, E) and P (x, E). These functions, given by 
eqns (9), can be computed directly from the characteristic of the 
non-linear element. This approach appears to possess an 
advantage over the original expression given by eqns (4) and (5). 
As a matter of fact, by means of eqns (11) a general method of 
machine computation of the describing function can be developed. 

From the conceptual point of view, eqns (11) are interesting 
by themselves. With each single or double-valued non-linearity 
can be associated two single-valued functions which give the 
complete information about the non-linear element, in the sense 
that those two functions are sufficient to compute the describing 
function. 


Non-uniqueness of the Inverse Describing Function 

Memory Type Non-linear Elements 

To show the non-uniqueness of the solution of the integral 
eqn (11) (first half) for the case of memory-type non-linear 


! Eqn (6) : 


g(E) = 
b(E) = 


7 zE 

-1 
7 lE 


F 1 (EcosP,E)cosfid[l+ 

Uo 

f 2 * 1 

F 2 ( E cos /?, E) cos jl d/? 

* n 

F ± (. E cos/?, E ) sin /?d/?-f- 

—0 

4/ 

In -| 

F 2 (E cos /?, E) sin ji d/i 


t Eqn (8): 


^ P(E cosy, E) cosy dy+^j* Q (E cos y, E) cosy dy 

1 


b(E)=- 


nE 


L J o 


P (E cos y s E) sin y dy + Q(E cos y } E) sin y dy 


( 6 ) 


(«) 
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elements, it is sufficient to show the existence of a set of functions 
<2 0 (x, E), not identically zero, whose corresponding g ( E ) are 
identically zero. Assume Q 0 (.x, E) to be of the form 


Qo (*, E) = h 1 (x)m 1 (E) + h 2 (x)m 2 (E) (12) 


and attempt to choose h x (x), h 2 (x), m 1 (E) and m 2 ( E ) in order 
to have g (E) = 0. Substituting eqn (12) into the first half of 
eqn (11) 


»»i (£) 


E xh x (x) 
o (E 2 —x 2 ) i 


dx-hm 2 (E) 


E xh 2 (x) 
o ( E 2 -x 2 f 


dx = 0 


(13) 


This means that if h 2 (x), m 1 (E) and m 2 ( E ) are chosen arbitrarily 
(assuming that m 2 (E)lm 1 (E) has meaning) h (x) will be given 
by the solution of the following integral equation 


In Figure 1 is represented the block diagram of this non-linear 
element. 

In an analogous manner the non-uniqueness of the solution 
of the first half of the integral eqn (11) can be demonstrated. 


H3- 


Muitiplier 




Figure 1. Example of a non-linear element with a describing function 
identically null 


E xh x (x) 
o (E 2 -x 2 f 


dx= — 


m 2 (E) 

m i (E)J 


E xfo 2 (x) 
o ( E 2 -x 2 f 


dx 


(14) 


Solving eqn (14) for h x (x) (see Appendix) yields 


, , . 2d 


dz 


zm 2 (z)yh 2 (y) 


o m 1 (z)(x 2 — z 2 )*(z 2 — y 2 ) i 


d y (15) 


Therefore to every function of the type * 


Non-memory Type (possibly double valued) 

If the non-linear element is of the non-memory type, eqns (11) 
are reduced to 


. 


xQ(x) 

0 (E 2 —x 2 ) i 
Ce 

P (x) dx 


dx 


( 22 ) 


Q 0 (x, E) = h 2 (x) m 2 (E) 
2 m 1 (j E) d 


kx dx 


\ X yh 2 (y)[ X — 

Jo J y 


zm 2 (z) dz dy 


(z)(x 2 -z 2 f(z 2 -y 2 f 


(16) 


will correspond g (.E) = 0. 

To illustrate the procedure one example should be considered. 
Let 

m 2 ( E) 


Eqn (16) becomes 
Q 0 (x,E)=h 2 (x) m 2 (E) 
2m x (E) d 


m l (E) 


=E Z 


nx dx 


yh 2 (y) [ —— 2 f - 2 — jri dzd y ( 1? ) 

Jo Jy (x 2 -z 2 ) i (z 2 -y 2 )* 


but 


(. x 2 -z 2 ) i (z 2 -y 2 ) i 


Substituting (18) into (17) 

<2o (x, E) = h 2 (x)m 2 (E )-~ 


y (x 2 -z 2 ) i (z 2 -y z ) i 
d z = ~(x 2 + y 2 ) 


(18) 


yh 2 (y) (x 2 + y 2 )dy 


which can be reduced to 
>0 (x, £) = m, (E) (x) (E 2 -x 2 )- J' 


yh 2 (y)dy 


(19) 


( 20 ) 


Suppose that m x (E) = 1 and h 2 (x) = x then eqn (19) becomes 


Q 0 (x,E) = xE 2 -- T x i 


( 21 ) 


Both integral equations are Volterra integral equations of 
the first kind and their solutions will be uniquef. Therefore, given 
the functions g (E) and b (E) there will exist one and only one 
pair of functions Q (x) and P (x) that, substituted in eqn (22), 
will transform these equations in an identity. However P (x) 
and Q (x) are not sufficient to determine the non-linearity. As 
a matter of fact, the equation of the non-linearity will only be 
determined if P x (x), P 2 (x), Q x (x) and Q 2 (x) are known. From 
eqn (9) it can be shown that, given P (x) and Q (x), any set of 
equations P x (x), P 2 (x), Q x (x) and Q 2 (x) that satisfy eqn (9), will 
generate a different non-linearity with the same describing func¬ 
tion. Therefore, the knowledge of the describing function of a 
non-linear element is not sufficient to determine the equation of 
the non-linear element. Even in the case of single-valued non¬ 
linear elements, g (E) is not sufficient to determine the non¬ 
linearity. However, if in addition the even harmonics of the 
Fourier series representation are zeros, the non-linear element 
is symmetric and g(E) is sufficient. 


Inverse Describing Function 


The solution of the pair of integral equations which represents 
a non-linear element of the non-memory type is not far to seek. 
In the first half of eqn (22) let v ~ x 2 and y\ = E 2 , 

™7g(V>?)= [ dv (23) 

J ° (rj — p)* 


which is Abel’s equation*. The solution of (23) is well known 8 , 
and may be written in terms of the original variables as 


600 = 


% d [z 2 g(z)] 
o (x 2 -z 2 f 


( 24 ) 


* Because of the symmetry of eqn (13) the sub-indices 1 and 2 can 
be interchanged in eqn (16). 


t Q(x) and P(x) are assumed to be continuous functions. 
* Pointed out to authors by W. M. Wonham. 
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The solution of the second half of eqn (22) may be written direct¬ 
ly, provided d/dx b (x) exists, as 

71 d r 21 


P(x)= -Td^ [x2fe(jc)] 

Examples 

(1) Consider the following describing function 
g(£)=2-E 2 ; b(E )=0 


(25) 


(26) 


A more convenient form perhaps for calculation of eqn (24) is 

(27) 

{x-~z~r 

which for this example becomes 




Eqn (28) 


and if F (x) is assumed to be single-valued and symmetric, then 

y Q 0 )=Qi (*)= 0.2 00 =/i 00 = f 2 00 = f ( x ). 

Therefore 

/(x) = x 3 

(2) Consider the following describing function 
g(£) = 0 (£<u) 

4M 


(29) 


g(E)=^(E 2 -a 2 ) i (E>a) 


(30) 


b (£)=0 

In order to apply eqn (24), write 


Therefore 


and 


d 0 2 g(z)]=—--- 


dz 


G(*)= 


4 M 


zdz 


a (z 2 -a 2 f(x 2 -z 2 f 


From eqn (25), 


Q(x)=2M (,x>a) 
2(x)== 0 (x < cl) 

F(x) = 0 


(31) 

(32) 

(33) 

(34) 


If the non-linearity is assumed to be symmetric and single¬ 
valued it may be shown that, 


f(x)=^^=Mu(jx\ — a) sgnx 


(35) 


where u (x) is the step function. 

In Figure 2, eqn (35) is plotted. As could have been foreseen, 


the non-linear element which has the describing function given 
in eqn (30) is a relay with dead band. 

Table 1. Inverse Describing Function of the Describing Function Shown 
in Figures 3 and 4 

(Experimental and theoretical results) 


X 

*i(x) 

F,{x) 

Experimental 

Theoretical 

Experimental 

Theoretical 

-5-00 

-4-0017 

-4*00 

-3*9972 

-4*00 

-4-80 

-3*9920 

-4*00 

-4*0045 

-4*00 

-4-60 

-3*9971 

-4*00 

-3*9993 

-4*00 

-4*40 

-4*0047 

-4*00 

-3*9921 

-4*00 

—4*20 

-3*9951 

-4*00 

-3*9954 

-4*00 

-4-00 

-3*9912 

-4*00 

-4*0127 

-4*00 

-3-80 

-3*6030 

-3*60 

-3*6052 

-3*60 

-3-60 

-3*2020 

-3*20 

-3*2079 

-3*20 

-3*40 

-2*8027 

-2*80 

-2*9505 

-2*95 

-3-20 

-2*4020 

-2*40 

-2*8516 

-2*85 

-3-00 

-2*0034 

-2-00 

-2*7500 

-2*75 

-2-80 

-1*6020 

-1*60 

-2*6537 

-2*65 

-2*60 

-1*2025 

-1*20 

-2*5528 

-2*55 

“2-40 

-0*80520 

-0*80 

-2*4521 

-2*45 

-2*20 

-0*40372 

-0*40 

-2*3542 

-2*35 

-2*00 

-0*00499 

0-00 

-2-2475 

-2*25 

-1-80 

-0*00130 

0-00 

-2*1518 

-2*15 

-1*60 

-0-00100 

0-00 

-2*0517 

-2*05 

-1*40 

-0*02379 

0*00 

-1*8228 

-1*80 

-1*20 

-0*00046 

0-00 

-1-4007 

-1*40 

-1*00 

-0-00082 

0*00 

-1*0008 

-1*00 

-0*800 

-0*00140 

0*00 

-0-60107 

-0*60 

-0*600 

-0*00202 

0*00 

-0*20202 

-0*20 

-0*400 

0*0000 

0*00 

0*0000 

0*00 

-0*200 

0*0000 

0*00 

0*0000 

0*00 

0*00 

0*0000 

0*00 

0*0000 

0*00 

0*20 

0-0000 

0*00 

0*0000 

0*00 

0*40 

0*0000 

0*00 

0*0000 

0*00 

0*60 

0*20202 

0*20 

0*00202 

0*00 

0*80 

0*60107 

0*60 

0*00141 

0*00 

1*00 

1*0008 

1*00 

0*00082 

0*00 

1*20 

1*4007 

1*40 

0*00046 

0*00 

1*40 

1*8228 

1*80 

0*02379 

0*00 

1*60 

2*0517 

2*05 

0*00100 

0*00 

1*80 

2*1518 

2*15 

0*00130 

0*00 

2*00 

2*2475 

2*25 

0*00499 

0*00 

2*20 

2*3542 

2*35 

0*40372 

0*40 

2*40 

2*4521 

2*45 

0*80520 

0*80 

2*60 

2*5528 

2*55 

1*2025 

1*20 

2*80 

2*6537 

2*65 

1*6020 

1*60 

3*00 

2*7500 

2*75 

2*0034 

2*00 

3*20 

2.8516 

2*85 

2*4020 

2*40 

3*40 

2*9505 

2*95 

2*8027 

2*80 

3*60 

3*2079 

3*20 

3*2020 

3*20 

3*80 

3*6052 

3*60 

3*6030 

3*60 

4*00 

4*0127 

4*00 

3*9912 

4*00 

4*20 

3*9954 

4*00 

3*9951 

4*00 

4*40 

3*9921 

4*00 

4*0047 

4*00 

4*60 

3*9993 

4*00 

3*9971 

4*00 

4*80 

4*0045 

4*00 

3*9920 

4*00 

5*00 

3*9972 

4*00 

4*0017 

4*00 


Eqn (28): 


Q(x) = — dx 

x 


(* 2 - z y 2(x 2 - 

C ~T" /•> 


3 

2^ 


■n 


-x 4 (x‘ 


-d 


= 2x 3 


(28) 
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ON THE INVERSE DESCRIBING FUNCTION PROBLEM 


(3) A numerical method to perform the operations indicated 
by eqns (24) and (25) has been proposed 4 . The philosophy of this 
method consists in dividing the interval in which the describing 
function is known in sub-intervals and approximating g (. E) 
and b (. E ) by a polynomial in each one of these sub-intervals. 


solutions are given. However, the operations expressed in 
eqns (24) and (25) are in general very difficult to perform in 
practical cases, if not impossible. In addition to this analytical 
difficulty, it should pointed out that in the majority of practical 
cases the describing function is not given by an analytical ex- 



In the work by di Tada 4 a second degree polynomical was used for 
the differentiations, and a linear approximation for the inte¬ 
grations. 

In Figures 5, 4 and 5 one example is shown. In Table 1 the 
experimental and theoretical results are listed. 



Figure 3. Real portion of the describing function of an unknown 
non-linear element 


Figure 5. Calculated results for the non-linear element whose describing 
function is given by Figures 3 and 4 

pression, but rather is given by experimental data. For these 
reasons it is suggested that numerical computation procedures be 
used in practice. Such numerical procedures are presently 
available 5 . 
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Figure 4. Imaginary portion of the describing function of an unknown 
non-linear element 


Conclusions 

An analytical approach to the inverse describing function 
problem has been proposed. The problem is formulated in terms 
of certain integral equations of the Volterra type. The required 


Appendix 

The solution of Abel’s integral equation 


Fix) 

may be written as 


-d z 0<a<l 


= t'-JVL , 

J o (x 2 — z z ) a 
2 sin our d f* xF(x ) . 


Therefore for 


a) 

( 2 ) 


f(z) = zh 1 (z) 


i: 


nh 2 in) 


drj 


Eqn (2) becomes 


M*)=- 


7ZX 


”»iWjo (x 2 -rj 2 f 

A p dz f*_ zm 2 iz)r t h 2 iyi)dr; 

dx J o J o m l l 


( 3 ) 


(z) (z 2 — j? 2 )* (x 2 — z 2 f 
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J. E. GIBSON AND E. S. DI TADA 


DISCUSSION 


Author’s Opening Remarks 

Between the submission of this paper and the I.F.A.C.Congress we 
have prepared standard FORTRAN programmes for the describing 
function and inverse describing function computations based on the 
principles given in the paper, and these routines are in regular use in 
our laboratory. 

L. R. Young, Massachusetts Institute of Technology, 77, Massachusetts 

Avenue, Cambridge 39, Mass., U.S.A. 

The authors are to be congratulated for their presentation of a new 
and computationally simpler method of describing function determi¬ 
nation, and exposition of the equations for determining the form of 
a non-linearity by means of the inverse describing function. The latter 
technique should prove valuable in experimental identification of non¬ 
linear control elements, especially in biological control systems. Bio¬ 
logical adaptation rarely permits the use of static test inputs, but 
sinusoidal signals and harmonic analysis may be useful in exploring 
the non-linearity through its inverse describing function. 

Some questions arise concerning the limitations of the method 
discussed in the paper. First, the exclusion of all but gain type non- 
linearities from consideration eliminates the important class of prob¬ 
lems in which the frequency-dependent dynamics cannot be separated 
from the non-linearity in obtaining experimental data. Was this ex¬ 
clusion made for mathematical expediency or because of a fundamental 
limitation in the use of the inverse describing function with frequency- 
dependent non-linearities ? 

Second, does the existence of a memory type non-linearity with 
non-unique describing function imply that no useful information (in 
terms of an inverse describing function) can be derived for any memory 
type non-linearity ? 

Finally, is it always necessary to make some assumption about the 
symmetry or type of asymmetry of the non-linearity in order to de¬ 
termine a unique equation of the element from its describing function ? 

J. E. Gibson, in reply 

Dr. Young’s remarks on the usefulness of the inverse describing func¬ 
tion in the analysis of biological systems are appreciated. Let us make 
clear the classes of non-linear elements to which our approach applies. 
For computation convenience, we exclude those elements which are 
dependent on frequency. By memory-type elements we mean those in 
which the non-linear characteristic is a function of the amplitude of 
the driving signal. An example which comes to mind is the hysteresis 
loop of magnetic material. Here the hysteresis loop size depends on 
the amplitude of the driving signal. By non-memory elements we mean 
those with a fixed characteristic, for example as shown in Figure 5 of 
the paper, which may be multi-valued but which do not depend on 
the amplitude of the driving signal. This is not usually referred to as a 
gain-type non-linear element. 

The fact that the inverse describing function in certain cases is not 
unique certainly does not imply that it is useless in those cases. As 


shown in Reference 5 of the paper, one additional even harmonic de¬ 
scribing function term is usually all that is required to make a unique 
identification of the non-linear element. This same approach applies 
if the element is not symmetrical. 


Ya. Z. Tsypkin, Institute of Automatic and Telemechanics, Kalanchevs - 
kaia 15a, Moscow, 1-15, U.S.S.R. 


The paper shows that the accurate determination of the characteristics 
of non-linear element by means of describing functions is dependent 
on the solution of the integral equation, which, in most cases, is some¬ 
what slow. This task may be quite easily solved on the basis of the 
approximate expression of describing functions as given in References 
1 and 2. For the single-valued characteristics of the non-linear element, 
we get 

(i) 

Substitute under (1) A for T/2* {k = 0,1,2...) and multiply it by (—1 )*, 
there will be a series of equations. Starting from the addition of these 
equations, we obtain 



f 1 (^)=E(-i) 4 ^ r g 

k = 0 2 1 



This formula is very easy to calculate owing to the rapid convergence 
of the series. 

From the table 1,2 it is, in the same way, possible to get the ex¬ 
pression F(A) for more difficult cases. 


References 
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No. 8 (1958) 


J. E. Gibson and E. I. di Tada, in reply 

We thank Professor Tsypkin for mentioning his early work on the 
approximate solution of eqn (11) of the paper. This work was brought 
to our attention only a few weeks ago by R. A. Johnson, and definitely 
deserves to be cited as a reference. We should also cite Mathews 1 for 
his approximate solution of the form given in eqn (10) of the paper. 
As we remark in the text, this is a convenient method of calculating 
describing functions. In this paper, however, we concentrate our atten¬ 
tion on the inverse problem. 
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Relative Stability of Oscillations in Non-linear Control Systems 

Z. BONENN 


Summary 

The relative stability of oscillations in non-linear systems is investigat¬ 
ed using the incremental input describing function. The results can 
be used to obtain the response to small step inputs, and the method is 
illustrated by an example of an ‘oscillating servomechanism’. 

General formulae are given for calculating the incremental input 
describing function. 

Sommaire 

La stability relative des oscillations dans des systemes non lineaires 
est examinee 4 l’aide de la fonction descriptive a signal d’entree 
echelonne. Les resultals peuvent Stre utilises afin d’obtenir la reponse 
aux fonctions l’echelon de petites amplitudes. La methode est illustree 
par exemple d’un «servomechanisme oscillant». 

Des formules generates, permettant de calculer la fonction des¬ 
criptive a signal d’entrde echelonne, sont donnees. 

Zusammenfassung 

Die relative Stabilitat von Schwingungen in nichtlinearen Systemen 
wird mittels der „inkrementalen 44 Beschreibungsfunktion untersucht. 
Die Ergebnisse konnen dazu dienen, die Antwort auf kleine Sprung- 
anregungen zu bestimmen. Als Beispiel zur Erlauterung dieses Ver- 
fahrens dient ein „schwingender Regelkreis 44 . 

Allgemeine Formeln zur Berechnung der inkrementalen Be¬ 
schreibungsfunktion werden angegeben. 


Introduction 

The stability of oscillations in non-linear systems may be in¬ 
vestigated in various ways. One method, suggested by West 1 * 2 , 
is to assess the stability according to the Nyquist criterion by 
plotting the open-loop frequency response locus for an in¬ 
cremental signal. The application of this method requires prior 
calculation of the incremental input describing function which 
may be calculated in a straightforward manner from the usual 
describing function 3 ’ 4 . 

In this paper, it is shown that by application of the describing 
function for an incremental input it is possible to obtain the 
stability and also the relative stability of oscillations. This solu¬ 
tion can be used to obtain the response to small step inputs 
which may be considered as a perturbation to the steady-state 
oscillation. 

Theory 

The standard mathematical approach to the stability problem 
of periodic oscillations in a non-linear system may be sum¬ 
marized as follows 5 . Suppose the system is represented by the 
differential equation (Figure 1) 

x + G(D)-f(x) = 0 (1) 

where G(D) is a linear differential operator with D = d/dt and 
fix) is a non-linearity. When eqn (1) has a periodic steady-state 


solution x ss at frequency w s , the stability of this solution may 
be investigated by forming the first variation of eqn (1) with 
respect to x 5S . This variation is given by 

Ax + G(D)[^f-) -Ax=0 (2) 

\ax/ x=Xss 

Ax is the perturbation, the behaviour of which will determine 
the stability of x ss . 

Eqn (2) is a linear differential equation with periodic coeffi¬ 
cients, the solution of which may be reduced to that of a set of 
linear algebraic equations. This set, however, is usually an in¬ 
finite set and its solution to a sufficient degree of accuracy is 
usually a lengthy and difficult matter. 



Figure 1. Non-linear system 


For on-off control systems the above approach has been 
reduced to a neat form by Tsypkin 6 , whereby the variational 
equation of the on~off system is that of a linear sampled data 
system, the stability of which may be determined by known 
techniques. This method becomes quite complicated when the 
periodic oscillations are not simple, furthermore it does not 
apply to continuous non-linearities. 

It appears that one of the main reasons for the practical 
difficulties in applying the above mathematical approach is that 
no restriction is placed beforehand on the perturbing signal, 
which may take any form whatsoever. 

The solution of eqn (2) may be simplified by assuming a 
specified form of incremental perturbing signal. This method is 
analogous to the solution of eqn (1) by the describing function 
method (where by assuming a simple form for the steady-state 
solution, namely 

x ss =aQ jw s t (3) 

and neglecting components of at other frequencies, the 
solution is greatly simplified). 

In the same manner, the solution of eqn (2) may be simplified 
by testing the stability of the steady-state oscillation with respect 
to a specified incremental perturbing signal 

Ax = Aa-e fft -e H(ot+y) (4) 

where A a = constant, co varies from 0 to oo and y is an arbitrary 
phase angle. Using the theorem 7 

G (D) Q at f {t) — Q at G (D + a) • / (t) (5) 

eqn (2) may be written as 

Aae J(c0,+y) + G(D + a)f^) -Aae J(a,+y) = 0 (6) 

X = Xss 
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The first term is at frequency co. In the second term, 


4 \f 


, , ■Aa-e J(a>,+r) 

\d.xJ x=Xss 

is the output of the non-linearity due to the incremental signal 
and contains components at co and at other frequencies. Here 
the usual describing function approximation is made by neglect¬ 
ing output components at other frequencies and replacing the 
output term by 

AaN i (a)Q J ^ wt+v) ( 7 ) 

Here N t {a) is the incremental input describing function which is 
defined by 


N — out P ut componen t at co due to Aa 
A a 


( 8 ) 


Nfia) is calculated for a (i.e. for the steady-state amplitude, the 
stability of which is under investigation). Note that A a ’and 
Ni(a) are constants. Hence, using eqns (6), (7) and (5) the 
following is obtained. 

1 + G (cr +jco) N t (a) = 0 (9) 

Eqn (9) is an extension of the equation used by West to determine 
the stability of oscillations. 


l + G(jco)-N i (a) = 0 (10) 

This extension is valid because eqn (10) is a linear equation 
[Ni(a) is a constant for the given (a)], hence G(jco) can be re¬ 
placed by G (a + jco). From the block diagram (Figure 2) corre- 



Figure 2. Incremental system 


sponding to eqn (9) it is seen that the effect of the non-linearity on 
the perturbing signal is expressed by the equivalent gain N t (a) m 
Also, it is obvious that the transition from, eqn (10) to eqn (9) 
is exactly analogous to the transition from stability determina¬ 
tion by the Nyquist criterion to the calculation of the closed-loop 
poles by the root-locus method. 


Eqn (9) yields n solutions for the closed-loop poles, a x + j co l5 
•••’ <*n +jo) n [n is the order of eqn (1)]. The complete solution 
for the perturbation is given by a combination of the n elemen¬ 
tary solutions whose relative amplitudes are determined by the 
initial conditions. The damping factors (or) indicate the rate of 
decay or otherwise of the initial perturbation and thus determine 
the relative stability of the steady-state oscillation. Also, the 
solutions of eqn (9) can be used to find the response to small 
step inputs which may be considered as a perturbation to the 
steady-state oscillation. 

Among the n solutions of eqn (9) there are at least two 
solutions at the steady-state frequency co = co s . One of these 
solutions is with a = 0. This solution, which does not decay, 
corresponds to a shift on the limit cycle and demonstrates the 
fact that the limit-cycle oscillation possesses only orbital stability 
but not asymptotic orbital stability 5 . 

As for the other solutions, due to the filtering assumption 
used in describing function analysis, only solutions with co < co, 
are considered. These solutions are always non-synchronous in 
odd non-linearities 8 . 

The practical application of this method depends on the 
ready availability of A^-. Simple general formulae which express 
Nt directly in terms of the ordinary describing function are given 
in the final section. Both forms of N h synchronous and non- 
synchronous, are required for the complete solution of eqn (9). 

Example-Oscillating Servomechanism 

Consider the system shown in Figure 3. Using conventional 
describing function analysis it is easily shown that the system 
has a stable steady-state oscillation E = a sin co s t where co s = 10* 

(;Figure 4). MacColl has shown 9 that these oscillations effectively 
linearize the relay for small, low frequency (compared with co s ) 
input signals. Hence, he designated such a system as an ‘oscillat¬ 
ing servomechanism 5 . 



Figure 3. Oscillating servomechanism 
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The response of this system to a small step input may be 
obtained by viewing this input as a perturbation of the steady- 
state oscillation which decays according to the roots of eqn (9). 
First obtain the solution in the synchronous case where eqn (9) 
becomes 

1 4- N is G (cr +j(D s ) — 0 (11) 


Using N is for the ideal relay [eqn (25)] two synchronous solu¬ 
tions are obtained (Figure 4) 

Si =.7(10*), 71 = 90°; s 2 = -0-7+; (10*), y 2 = 65° (12) 

The first solution corresponds to a phase shift of the steady-state 
oscillation which does not decay. The second solution has an 
angle y — 65° with respect to the steady-state oscillation. The 
third solution is given by 

1 + N ins G (<r +jco )= 0 (13) 


This is a non-synchronous solution. Note that for the ideal 
relay [eqn (25)] 


N„ 


N 1 

2 2 G(jco s ) 


(14) 


Hence [eqns (13), (14)] 

G(cr +jco) — 2\G(jco s )\ < — 180° (15) 

This equation is easily solved by root locus techniques yielding 


s 3 =-10-55 


(16) 


Cosgriff’s approach 12 is quite similar to the author’s, assum¬ 
ing the same type of incremental signal [eqn (4)] to be used in the 
solution of eqn (2). However, he does not use the concept of the 
incremental input describing function. This concept enables the 
author to formalize the solution [eqn (9)] and clarify it. Also, as 
Ni can be calculated very easily from the ordinary describing 
function the solution becomes a routine matter. 


The Incremental Input Describing Function for the Ideal Relay 

Synchronous Case 

The input to the non-linearity fix) is assumed to be 

x~ae iq + Aaz i(q+y) (21) 

where a is the steady-state amplitude and A a < a is the per¬ 
turbation. N is , the synchronous incremental input describing 
function, is given by 3 

N “= N+ f^ 1+e_2/V) (22) 

where N is the ordinary describing function. When N^ s is plotted 
in the complex plane as a function of y with a as parameter a 
family of circles is obtained. When y — 90°, N is = N. 

The solution of eqn (9) in the synchronous case yields at 
least two intersections between the — N is circle and 
l/G(c + jco s ). One of these must be, due to the steady-state 
condition, at N is = N, that is at a = 0, y = 90°. This solution 
corresponds to a phase shift of the steady-state oscillation. 


The complete solution for the perturbation is given by [eqns (12), 

(16)] 

AE(t) = Aa 1 *cos 10* t 

+Aa 2 e-°- 7t -sin(i0 i t+65°) + Aa 3 e- lo - 55t (17) 

Assume, for example, a small step input AR at t = 0. Using the 
initial conditions 

AE(Q) = AR; AE f (0) = AE"(0)=0 (18) 

AE{t) is given by 

(f) = /IjR [ — 046 cos 10* • £ 

+ l-5e _o ' 7 'sm(10*-f+65°) + 0- le -l0 ' 55t ] (19) 

The response is dominated by the first and second term, 
which represent the phase and amplitude perturbations of the 
steady-state oscillation. 

Relation to Previous Work 

Two other approaches to the problem of the relative stability 
of the steady state will be outlined. Starting from the con¬ 
ventional describing function solution 1 + N(a) G(ja) s ) = 0, 
Grensted 10 * 11 assumed that the perturbed oscillation is at a 
variable complex frequency near the steady-state frequency 

— fi+jcQ= jco s -hAz (20) 

From this assumption he derived a rather complicated varia¬ 
tional equation which he solved for the perturbation modes. 
Two solutions equivalent to our synchronous solutions are 
obtained. However, due to the above assumption [eqn (20)] the 
non-synchronous solutions cannot be obtained. 


Non-synchronous Case 

The input to the non-linearity is now assumed to be 


x — aQ ip + Aac iq 


(23) 


In this case the phase angle between the steady-state oscilla¬ 
tion and the perturbation varies continuously. Hence, N { for the 
non-synchronous case (N ins ) is obtained 4 from N is by averaging 
over the phase angle y. 


at a dN 

lf t" maN+ Y'da 


(24) 


Ni for the Ideal Relay 


Applying the above equations it is found that JV$ for the ideal 
relay is given by 


tf is = y( l-e" 2jY > 



(25) 
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DISCUSSION 


Z. Bonenn, Scientific Department , Ministry of Defence, P.O.B. 7063 
Hakirya , Tel-Aviv, Israel 

Author’s Opening Remarks 

Nature of the Approximation 

Further work was carried out to verify more closely the approximate 
solution presented in the paper. For an odd non-linearity the steady- 
state solution will have components at co s , 3 co s , 5 co s ... Using the 
usual describing function approximation components at co ^ 3 co s are 
neglected in the steady solution. When the incremental input is added 
to the steady-state oscillation 

x = a sin co s t+A a sin (cot+y) 

Output components will appear at co and 2 nw s ± co (odd non- 
linearity) « = i, 2 ,... 


2d s -u) 2o s *u 4 cj s -o 4 < j 5+ cj 
Figure A 



Hence, the general form of a solution of eqn ( 2 ) is 


Ax = e a? 


Aa 0 sin (cot + y 0 ) 


00 


+ E{Aa n sin [(2 noj s + co)t + y„] 


1 + 


1 . d N 
" da 

After one iteration we find 


A T ;„o- ja- 



-G((r+j 2(o s ) = 0 


Ax = Aa 0 -e 1 °' 5t [1 - 0,12 • sin (2 co s t +61°)] 

Thus we see that the approximation is well justified whenever the 
usual describing function approach is valid. 

The validity of this approach may also be seen by comparison 
with the work of Dr. Gelb on limit cycling control systems published 
recently in Trans. I.E.E.E. April 1963. Dr. Gelb derives heuristically 
a second-order differential equation for the dynamics of the limit 
cycle, from which he calculates transient behaviour of the steady-state 
oscillation under test. Checking experimentally the system shown in 
Figure B, he found good agreement for small f. However, for large f 
there is a considerable difference between theory and experiment 
For example, for 



Figure B 


Dr. Gelb's results are ; Theoretical ■ 


3-22 


Our method gives; 


Experimental; 


2-25 

2-36 


+ A a n sin [(2 nco s — co) t + y„]} 


In line with the usual approximation we may neglect components at 
4 ai s ± co and above. Only the components at 2co s ±co must be 
investigated. In the synchronous case co = co s and 2 <o s ±co->co s 
3 co s . The component at co s resulting from 2 co s — co s is taken into 
account m N ts —the synchronous incremental describing function in- 
deed it is this component which makes N {s complex and of circular 
lorm. The component at 3 co 5 is neglected as before. 

form 16 ° ther S ° lutl0n in our exam P le is mainly aperiodic and of the 

Ax = e at Aa 0 [l+£ 1 sin(2m s t + y 1 )-K..]m = 0 

Neglecting 2 w s 0 ) and using eqn (13) 


1 + A';„ s G (ff) = 0 

we found a = — 10*55 

The second approximation is obtained by solving 


1 + 


Ni ns +~a\ — 


e i „l &ReN . d ImN 


da 




d a 


— cos^ 


G (o) = 0 


R J ^ 0 ^ HENBURGER ’ Universif y °f Connecticut , Storrs, Connecticut , 

Dr. Bonenn has contributed much to the concept of the incremental 
describing function through this and his earlier papers. As he illus¬ 
trates in the example, the method is an ingenious and useful one when 
investigating the degree of stability associated with small excursions 
from steady-state conditions and when dealing with an oscillating or 
limit-cycling type of system. 

I attempted to extend, on my own, the author’s technique to either 
what probably is the more usual situation when a non-oscillating 
steady-state response is the goal, or to the case of large excursions. 
I can report no success when using this particular technique. Could 
the author make some suggestions? For the example he treats, how 
would he consider a contactor with an inactive zone large enough to 
prevent oscillation ? The best technique I was able-to find for the prob¬ 
lem just mentioned, was resorting to the usual describing function plot 
and superposing this on a family of loci of the function 

G 1 (— (pa) 0 + j ]/l —q?(o 0 ) 

A plot then appears as shown in Figure A. When operating points 
for negative values of cp exist for the lower amplitudes, as they do in 
the example, a limit cycle is indicated. The time required to settle to 
such a steady-state condition, or the settling time for non-oscillating 
systems, may be reasonably approximated. 
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G-'(S) plane 



In any event, comparisons drawn between various modifications 
of the describing function technique are meaningless unless accom¬ 
panied by actual response examples on real systems or their analogue 
simulation. As a general suggestion for sessions on this subject, I 
would like to propose such a type of verification whenever it is pos¬ 
sible. It is likely that such a procedure would have demonstrated the 
validity of Dr. Bonenn’s interesting technique when applied to many 
practical problems. 


Z. Bonenn, in reply 

I thank Professor Kochenburger for his interesting discussion and 
quite agree that the method is valid only for small excursions from 
the steady-state oscillation. I am afraid I can offer no solution for 
large excursions or transient conditions. 

Dr. Grensted (Reference 11 of my paper) attacked this problem, 
but his approach becomes very complicated and difficult to apply for 
systems above second order. 

I hope that my previous remarks on the accuracy of the approxi¬ 
mation and the comparison with Dr. Gelb’s experimental work 
demonstrates the validity of this technique. 


J. L. Douce, The Queen's University, Belfast, Northern Ireland 

The method is limited by the restriction that not only the perturbation 
but also the response to the perturbation must be small. This means 
that the complete response cannot be determined if the final state 
differs appreciably from the initial state in amplitude or frequency. 
Practical applications require an investigation of the stability of forced 
oscillations considered in the paper. This investigation must consider 
the possibility of the jump effect and the occurrence of subharmonic 
oscillations. Equation (10) can be used for this purpose. Does the 
author consider that eqn (9) has any advantage in this application? 

I would suggest that a more usual definition of the describing func¬ 
tion is the negative reciprocal of the author’s N. The figure correspond¬ 
ing to Figure 4 now consists of the open-loop frequency response 
G (jo>) and — 1 /IV. I found the labelling of this figure confusing, due 
to this consideration. 

The general expression for the incremental describing function, 
eqn (22), was, so far as I know, first published in 1955. 


Z. Bonenn, in reply 

I quite agree with Dr. Douce that for calculating the jump region in 
the case of forced oscillations eqn (10), as shown by Professor West, 
should be used. 

As for using N or—1 jN, this is a question of taste and convenience. 
I use both. 

I shall be grateful to Dr. Douce if he will give the 1955 reference 
to eqn (22). This previously was unknown to me. 


T. F. Joos de ter Beerst, Universite Lovanium, B.P . 214, Leopoldville 
XI, Congo, Africa 

I should like to have some more details about the use of this theory 
when the non-linear function is piece-wise continuous and, in partic¬ 
ular, of the use of the variational equation to establish the stability 
even in the case when /(x) is not everywhere continuous. 

I would also like to point out that in some cases, the perturbation 
should have two terms instead of one, as stated in eqn (4). This can 
be shown using Floquet’s theory, as will be shown in a paper to be 
published soon. These cases arise when the non-linear feedback system 
is able to oscillate freely without external excitation. If one studies the 
stability of this free oscillation, or of an externally excited oscillation 
at a frequency near that of the free oscillation, then the perturbation 
should be written 

Ax=Aa^ +Ja)t + Ab^- Ja)t ( 1 ) 

where A a and A b are two complex quantities. The phase-shift y of 
eqn (4) of the paper has been included directly in A a. 

That there should be a second term in Ax can be shown as follows. 
Among all terms at the output of the non-linearity, there will be one 
at a frequency — to, and this term is not filtered out, if |o"l is small, 
which is implicitly admitted by the author since the term with fre¬ 
quency co is not filtered out. Using Ax in (1) as perturbation, eqn (6) 
becomes 

Aae Jat +Abe~ Jm 

+ G(D + Jf) [Aa e J<ot +Ab e ” J<ot =0] (2) 

\dX/ X = Xss 

One develops (^pj in a Fourier series and obtains terms at 
frequencies 0,2 co and — 2 co, and other terms 

=a 0 + a 2 e 2j<ot + a 2 e~ ljoJt + ... 

Replacing in (2) 

Aa e'“" + A b + G(D + a) [(a 0 -Aa + a 2 -Ab) e Jat 


+ (a 0 -Ab + a 2 -Aa)e Jat + ... =0 


where the neglected terms have a frequency different from ±o>. 
Grouping terms, equating separately the coefficients of e ?Ct)i and e 3(0 
to zero 

f Aa + G (cr +jco) [a 0 * Aa + a 2 * Afr] = 0 
(A b + G ( <7 —jco) j> 0 • Afr + a 2 * Aa] = 0 

For this system to have a solution in A a, A b different from zero, the 
following determinant should be zero. 


l + a 0 G(cr +jco) 
\a- 2 G(a—jco) 


a 2 G{a +jco) 
1 +a 0 G(a-jco) 


=0 


(3) 


This equation replaces eqn (9) of the paper. In addition it can be shown 
by Floquet’s theory that cr should have a small magnitude and that it 
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is therefore allowed to develop G (a + jw) and G (a—jm) in a Taylor 
series keeping only the first two terms 


G(a + jco) = G(jco) + a(^tl 


G(<7-jco) = G(-jco) + J^^] + 

Thus eqn (3) is always of the second degree in o, and at most two 
stability conditions have to be met to ensure that the real part of o 
should be negative. 
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Z. jBonenn, in reply 

I thank Mr. Joos de ter Beerst for his interesting discussion. However 
I cannot agree with his remarks. Indeed, I cannot see why the two 

frequency^. 0) n0t be COmbined into ™ *£»at 

It seems to me that this difference of opinion arises as follows 
Going back to the general form of the perturbation we observe (see 


term^YY that aS the P^turbation consists of two 

Ar n Z and 2as ~ coorlf we wish w 5 - A w and w s + Aw When 
Aw 0 these two terms coalesce, it is this coming together of the two 
errns which creates the synchronous incremental describing function 
which is complex and of circular form. Hence, when using our eq n (9) 
with N( S we take care of both terms. q 

‘almost S ' tUa , tion is Y® differ ent when Aw #= 0. This is the difficult 
almost synchronous case. Mr. Joos de ter Beerst’s equations [before 
(3)] represent this case. Noting that 4 1 


a 0 = N im 


Ct 2 —--— * 


a dN 


a_ 2 = — 


a_'d& 

2 do 


equationsforth!: if Y C ° n ? 1 “’ hiS equations correspond to our 
equations for the almost synchronous’ case. 


da J 


G [a +j (co s + Aa>)] = 0 


diffiSt to fnlffi t h6Se equatl0ns ft tUnlS 0Ut that i£ is ^remely 
solution In£2 ill n ® cessar y conditions for their simultaneous 

has such iS n0t yet SUCCeeded in findin 8 a «y astern which 
has such a solution. Moreover, it is easy to see that such solutions are 

m sec ° nd_ and third-order systems because there must be 
at least two synchronous solutions. I have tried to obtain an almost 

Here h I°obteined U e th 'Y fourth :° rder system ’ but without success. 
Here 1 obtained either two synchronous and two non-synchronous 

solutions or four synchronous solutions y 
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Predictive Control of an On-Off System with Two Control Variables 

A.J. ADEY, J.F. COALES and J. A. STILES 


Summary 

This paper studies the optimum control of multi-variable systems by 
considering the behaviour of the two-variable on-off system described 
by the equation 

x -f Ax — u 

where u l9 u 2 = ± 1. 

By the application of Pontryagin’s maximum principle, it is shown 
how the number of switchings and the drive ratio uju^ for an optimum 
trajectory (minimum settling time criterion), depend on the nature 
of the matrix A. For the case when the interaction terms a 12 and a 21 
have the same sign, the maximum number of switchings for an 
optimum trajectory is three, and the drive ratio uju 2 near the origin 
of the error phase space has the same sign as a 12 and a 21 . 

A predictive controller described in this paper finds the three 
switching times by an iterative process which involves the application 
of logical rules to the predicted behaviour of the system, computed in 
a fast analogue model. This realizable controller is optimum when 
the two-by-two matrix A has a 12 a 21 > 0; and will also control systems 
where a 12 a 21 < 0, although not optimally. It is also shown that the 
problem of hunting may be overcome by confining the hunting to the 
fast model alone. 

Sommaire 

La pr6sente communication 6tudie la commande optimale de systemes 
a variables multiples en considerant le comportement du systeme par 
Tout ou Rien k deux variables, dtait par l’dquation x + Ax = u 
(ou u 2 = ± 1). 

L’application du principe du maximum de Pontryagin montre la 
maniere dont le nombre de commutations et le rapport uju 2 pour 
la trajectoire optimale (critere du temps d’Stablissement minimum), 
d6pendent de la nature de la matrice A. Lorsque les termes d’inter- 
action a 12 et a 21 possedent le meme signe, le nombre maximum de 
commutations sur une trajectoire optimale s’&eve k trois et le rapport 
uju 2 , au voisinage de Torigine de l’espace de phases de l’erreur, 
possede le meme signe que a 12 et a 21 . 

Un regulateur k prediction, dScrit dans la prdsente communica¬ 
tion, trouve les trois moments de commutation au moyen d’un 
processus itdratif qui applique des regies logiques au comportement 
prSdit du systeme, calculi sur un modele analogique rapide. Ce 
regulateur realisable est optimal lorsque la matrice deux-par-deux A 
verifie la condition a 12 a 21 >0; il peut egalement commander — 
quoique non-optimalement — des systemes pour lesquels a 12 a 21 < 0. 
II est dgalement montre que le probleme d’auto-oscillations peut etre 
r£solu en limitant ces auto-oscillations au seul modele rapide. 

Zusammenfassung 

Der Aufsatz untersucht die optimale Regelung von Mehrfachsystemen, 
indem er das Verhalten eines Zweipunktreglers fur zwei Variable, das 
durch die Gleichung x A- x A= u, wobei u lf u 2 — ± 1, beschrieben 
wird, betrachtet. 

Die Anwendung des Pontrjaginischen Maximumprinzips zeigt, 
wie die Anzahl der Umschaltungen und das Antriebsverhaltnis 
fiir eine optimale Trajektorie (Kriterium der minimalen Einstellzeit) 
von der Beschaffenheit der Matrix A abhangt. Fiir den Fall, daft die 
Kopplungsglieder a 12 and a 21 das gleiche Vorzeichen haben, ist die 
maximale Anzahl der Umschaltungen fiir eine optimale Trajektorie 
drei und das Antriebsverhaltnis uju 2 hat in der Nahe des Ursprungs 
des (Fehler-)Phasenraumes das gleiche Vorzeichen wie a 12 und a 21 . 


Ein in dieser Arbeit beschriebener Regler mit Vorhersageeigen- 
schaften ermittelt die drei Schaltezeitpunkte durch einen iterativen 
Vorgang, der die logischen Gesetze auf das Vorhersageverhalten des 
Systems anwendet, das ein schnelles Analogmodell berechnet. Der 
realisierbare Regler arbeitet dann optimal, wenn in der quadratischen 
Matrix (2. Ordnung) A a 12 a 21 > 0 ist; es tritt auch eine Regelung von 
Systemen mit a l2 a 21 < 0 auf, diese ist jedoch nicht optimal. Es wird 
gezeigt, daB sich das Problem der stabilen Dauerschwingungen 
(hunting) ebenfalls dadurch bewaltigen laBt, daB man die stabilen 
Dauerschwingungen auf das schnelle Modell beschrankt. 


Introduction 

The practical design of control systems with more than one 
variable has generally used traditional linear methods. These 
methods often involve nullifying the effect of interaction be¬ 
tween the variables, which normally increases the complexity 
of the controller. However, in all practical systems either the 
input force (or torque) of the plant motor or the input power 
(or energy) to the plant is limited, and in many such cases 
it is known that the best control possible, in some sense, is 
obtained by using an on-off controller. Even if on-ofF control 
is not optimum, it will nevertheless usually be nearly so. 

When such controllers were first considered for single¬ 
variable systems, they were designed to operate by generating 
a switching function. However, in all but the simplest cases, the 
complexity involved in this is prohibitive. For this reason the 
method of predictive control was developed in the early 1950s 1 . 

In its first form this controller simulated repetitively the 
future behaviour of the system on a fast time scale. The in¬ 
formation provided by this simulation enabled the switching 
time giving the desired system behaviour to be found by a pro¬ 
cess of iteration. Subsequently, by using the general properties 
of the switching function, it has become possible to determine 
the correct drive direction by a single simulation 2 . A further 
development of these ideas is necessary for a satisfactory solu¬ 
tion of the two variable on-ofF control problem. 

The two-variable plant considered here is that described by 
the equation 

xA-Ax = u (1) 

where 



and u is constrained {Figure f). 

The more general system described by the equation 

y + Cy = Bu 

with B non-singular may be reduced to this form by putting 
y = Bx,A=B~ 1 CB 
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Such a system, i.e. second order with only inertia and viscous 
friction terms, forms the basis of many practical systems. 
Furthermore, the complexity of the controller increases rapidly 
with the number of variables and so it is necessary to under¬ 
stand the two-variable case before proceeding to more variables. 

This paper shows that, using minimum settling time as the 
performance criterion, the maximum number of switches for an 
optimum trajectory can be three, four or infinite, depending on 
the nature of the matrix A. If the interaction terms a 12 and a 21 
have the same sign, this number is three and the problem be¬ 
comes simpler to solve. A practical method is given for con¬ 
trolling a system of this type. 

Moreover, it is likely that, as in the single-variable case 1 , the 
same technique (of using three switches) can be used to control 
satisfactorily, though not optimally, this plant for any A, or to 
control two-variable systems with non-linearities or of higher 
order; even when the desired outputs are slowly varying random 
signals. Furthermore, following Chestnut 2 , it may only be 
necessary to use an approximate simulation of the plant, e.g. a 
second-order one. It is also shown how the problem of hunting 
may be solved and so the controller described in this paper is 
likely to have wide application. 

The Number of Switches 


(Hi) If 0 < a 12 a 2 1 , the number of switches can never exceed 
three (two in one component of u. and one in the other). 


These results do not apply to a small region of the state space 
where the system is singular i.e. ‘non-normal’. This region con¬ 
tains all points for which the optimum drive has no switch in 
one of the components of u* 

In order to prove (ii) and (///), which correspond to 2 X and 
A 2 real, only the case in which and l 2 are distinct and non¬ 
zero will be considered. The cases of equal eigenvalues or one 
zero eigenvalue may be similarly dealt with. 

In this case, the solution of eqn (2) may be written 


p = (A'- 1 W + 


-Aif 


/ p 


A x — A: 


(A'-aJ) a- 


(4) 


which shows that neither component of p can have more than 
two zeros. 

If a back-traced optimum trajectory contains four switches, 
at ^ and f 2 in u v and s 2 and s t in w 2 , there must be values of « 
and ft satisfying 

Pi(t 1 ) = p 1 (t 2 )=p 2 (s 1 )=p 2 (s 2 )=0 


For the system (1) with \u { \ < 1, and desired behaviour that 
the position and velocity should become zero in minimum time, 
it is well known how to ‘trace backwards’ optimum trajectories 
from the origin of the phase space 3 . 

For ‘back tracing’ the system equation becomes 

x—Ax=u 

The adjoint equation is 

p + A'p = 0 

This is integrated to give 

p + A'p — ft (2) 

where ft is a constant vector. 

The optimum drive is given by 

u = sgn p (3) 

Hence by choosing different initial condition vectors p (0) 
= a. and p (0) = ft — A'oc for the adjoint equation, different 
optimum trajectories are produced. 

To determine the complete optimum drive u(t) from eqns (2) 
and (3) one needs to know, first, when the zeros of the com¬ 
ponents of p occur, and, second, what the drives are for some 
given time, say t = 0. These two questions are considered in 
turn. 


These four equations, linear and homogeneous in the four 
components of oc and /3, have a non-trivial solution if, and only 
if, the determinant of the coefficients vanishes. This condition 
reduces to 


where 


^ 11 2.2 f(s I 5 S 2 ) 

e -A 1 x_ e -A 1 y 

/(x,k) = e - A2JC _ e - A - 


(5) 


This is therefore the necessary and sufficient condition that 
t l9 t 2 , s l9 s 2 should be switching times on some optimum tra¬ 
jectory. 

If l x > 2 2 , f(x,y) is non-vanishing, continuous, and tends 
to zero along the line x = y as x -> co. Thus eqn (5), being the 
quotient of two values of f(x,y), can take any positive value; 
i.e. by suitably choosing t l9 t 2 , s l9 s 2 , eqn (5) will always be 
satisfied provided a n — and a n — 2 2 have the same sign. If, 
however, they have opposite signs, or if one is zero, eqn (5) can 
never be satisfied. Substituting 

2i,2 2 =-y [<^ii + ^22 ±{( a u” a 22) 2 + 4«i2^2l}^] (6) 

(ii) and Qii) are proved. 


Zeros of p 

The explicit solution of eqn (2) depends on the nature of 
the eigenvalues 2 X , l 2 of A ; and on this solution depends the 
possible number of zeros of the components of p 9 i.e. switches. 
It is helpful to classify A by its cross-terms a 12 and a 21 as follows. 

(0 If a 12 a 21 < — {(fl u — a 22 )/2} 2 9 A x and l 2 are complex, and 
there is no upper bound to the possible number of switches. 

(//) If — {(a n — a 22 )j2} 2 < a 12 a 21 < 0, the number of switch¬ 
es can never exceed four (two in each component of u). 


Drive Directions 

The drive ratio u 2 /u x depends on the solution of eqn (4) and 
the cross-coupling terms a 12 and a 21 . 

If 0 < a 12 a 21 , viz. a type (Hi) system, this ratio always has 
the same sign at the origin of the state space, i.e. at t = 0 for 
‘back tracing’. 

* This region, as well as the boundaries in state space for the 
various types of drive, has been extensively analysed since the paper 
was written. 
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In order to prove this, consider the case of two switches at 
times t x and t 2 in u l9 and one switch at time s x in u 2 . The proof 
is similar when there are two switches in u 2 and one in u v 

If a 21 =|= 0 the ratio u 2 lu^ at the origin may be written 

a 2 _ ( a l 1 ~^l) ^1 (gzi ~~ gll) (^11 ^ 2 ) ^2 fell ~~ gn) 

a l ^21 (gll&22"”&12£2l) 

where 

gij= 1 —e~ A, ° and h t = l-e~ XiSl 

Since g X j/g 2 j — f (0, fy) is monotonic in t 5 the denominator 
can never vanish. Thus, as the ratio is a continuous function 
of t x , t 2 , s x , if its sign changes it must vanish for some positive 
^ hi s i- This means oc 2 = 0, i.e. that there is a fourth switch 
at the origin, which has just been shown to be impossible for a 
type (Hi) system. 

To see what this sign is, suppose X x > X 2 and let t x —> co. The 
ratio then becomes 

a 2i (e~ Xt ‘ 2 —e~ Xlt2 ) 
or 

( a n — Ax) h x 
aiigu 

Since for ct 12 a 21 ^ 0, by eqn (6), X x > X 2 implies X x > a n (and 
in this case a n — and a n — 1 2 have opposite signs), both 
these ratios have the same sign as a 21 ; i.e. the drive ratio at 
the origin has the same sign as the cross-coupling terms. 

For a forward-time optimum trajectory containing three 
switches, the starting drive ratio is therefore opposite in sign to 
the cross-coupling terms, provided a 12 a 21 > 0. 


On the other hand, if a 12 a 21 < 0 the drive ratio at the origin 
of the state space may be different for different trajectories. 

The controller described below has been developed for the 
case a 12 a 21 > 0. These results considerably simplify its con¬ 
struction: for, in searching for the optimum trajectory three 
times may be taken as parameters; and, of the four possible 
starting drives, only two are permissible. 

The Control Strategy 

A logical strategy has been developed for determining the 
correct drive vector to control the plant described by eqn (1). 
So far only the case with all the coefficients a llf a 12 , a 21 and a 22 
positive has been considered. The general layout of the plant 
and controller is given in Figures 1 and 2. Figure 1 shows the 
system with S as the Laplace operator, while Figure 2 shows the 
controller in which G gates the initial conditions for the model 
and s is the Laplace operator with respect to fast-model time. 

The behaviour of a trajectory is investigated by viewing the 



Figure 1 


if X 2 > 0 

if X 2 < 0 



Figure 2 
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two projections P 1 and i> 2 of the trajectory on the phase plane 
of error and its derivative for each of the two variables. This is 
satisfactory because, due to the nature of eqn (1), each output 
variable is affected most strongly by the corresponding drive; 
and in fact the projections P 1 and P 2 resemble the trajectories 
produced by single-variable second-order systems, although 
somewhat distorted by interaction. " 

These projections are generated repetitively on a fast time 
scale by an input calculator and an analogue model of the plant 
The optimum switching times are found in this fast model from 
arbitrary values by a process of iteration which uses the shape 
and position of the projections in one time sweep to find better 
switching times for the next time sweep, and when necessary to 
alter the drive directions. 

. general there are three switches, say for example two in 
drive % and one in drive u 2 . The shape of the projection P, 
indicates that one switch in drive u x , called the main switch may 
be adjusted to make the projection P x pass through the origin 
of its phase plane. Similarly the main switch in drive «„ may be 
adjusted to make the projection P 2 pass through the origin of 
its phase plane. It is also apparent that the other switch in 
drive u 1; called the supplementary switch, may be adjusted to 
make the two projections reach their respective origins at the 
same instant of time. 

As there are two switches in drive these may be adjusted 
to make the trajectory pass through the origin of the phase 
space no matter which direction the drive u x has initially. The 
initial direction which gives the shorter settling time is therefore 
chosen and this is found by experiment to be that which gives 
the initial drive ratio Ul (0)/u 2 (0) = - 1. This agrees with the 
earlier result (here a 12 and a 21 are both positive). 

The two main switching times can be adjusted simultaneously 
because their mteraction merely slows their convergence to 
steady values. However, interaction can prevent the supple¬ 
mentary switching time being adjusted to its proper value 
simultaneously with the main switching times; so it is necessary 
to keep the supplementary switching time fixed while the main 
switching times are converging to steady values. Whenever these 
values have been reached, the supplementary switching time is 
adjusted; after which the main switching times converge to new 
values. This process is repeated until the supplementary switch¬ 
ing time has also arrived at a steady value, when the model 
trajectory will be the optimum one, except in the small singular 
i.e. non-normal’, region of the state space. As no special provi¬ 
sion is made for this region, the control here is good but not 
optimum. 

. .. A , S , the d ' rectlons of the P lant drives are the same as the 
mtia 1 drive directions in the model, this system will cause the 
plant drives to be optimum except possibly during the iterative 
process in the model. This however only occurs after a signifi- 

hm m n n f m ^ d f Sir£d ° Utput and lasts for onl y a short time 
rLV f n C A ? thls system wher « each time sweep lasts a 
maximum of tm see and the basic plant time constant is lOsec). 

belovT* lt6ratlVe process 1S governed by the rules set out 

The Detailed Logical Rules 

r selm^r^VT T iS measured dur “8 each time sweep with 
lasts r - b t egU1nmg of the trajectory. Each time sweep 
until x - x f , a time greater than any possible settling time; 


and between each time sweep the initial conditions for the 
model and its drives, as well as those for the input calculator 
are set equal to the present conditions in the plant. The logical 
rules for adjusting the switching times, etc. are applied during 
this resetting period. 8 

At any instant, the controller operates in one of three pos- 
sible modes. These are: P 

Mode 1. Here the supplementary switch occurs in the drive u 
at time r\. 5 

Mode 2. Here there is no supplementary switch. 

Mode 3. Here the supplementary switch occurs in the drive u„ 
at time r 2 . 

In all modes the main switches occur at r x in the drive iq and 
at t 2 in the drive u 2 . If a supplementary switch does not exist 
the corresponding r' is zero in the calculations which follow’ 
e.g. in Mode 1, t 2 — 0 and in Mode 2, x\ — — 0 

Hereafter only Modes 1 and 2 are considered, because the 
rules for Mode 3 are found from those for Mode 1 by inter¬ 
changing the suffices 1 and 2 throughout. 

• 0r . convenience in the electronics, in Mode 1 the switches 

m the drive u x are ordered so that r\ < r 3 . Let: 

ei^Zi-x x 

where z is the desired output. 

CO— —Mi (t)* 6 1 (t) 
m 1 (x) = \e 1 (x)\+k 1 \e 1 (x)\ 

4i ( T )=sgn {—Mi (t) • (t)} 

where k x is a positive constant chosen by experiment. 

In each time sweep, the following test is made during the 
interval < r < tq (Figure 3). 

Is d x > e at the instant r ai when q x = + 1 for the first time ? 
v £ is a small positive constant chosen by experiment.) 



Figure 3 
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Depending on the result of this test, the following action is 
taken: 

If yes: 

(0 Store the function c x = d 1 (x ax ) for use under Rule 1. 

(ii) Set the resetting time x rx = x ax (see note on p. 46). 

(in) Apply Rule 3 after this time sweep. 

If no: 

Make the following test at the switching instant when x = t x . 
Does q x change from — 1 to + 1 ? 

If yes: 

(0 Store c x — m x ( x x ). 

(ii) Set x rx — t x . 

If no: 

Make the following test during the interval r x < x < Xf. 
Does the instant t& x at which q x = + 1 for the first time 
exist ? 

If yes: 

(0 Store c x = d x ( x bl ). 

00 Set x n = tfij. 

(Hi) Store the settling time r 8x = t& x (see below). 

If no: 

(0 Store c x = 0. 

00 Set r ri = Xf. 

(Hi) After the time sweep, find the new value for r x by setting x x 
to one tenth of its previous value. 

Similar tests are performed on the second variable by chang¬ 
ing the suffix 1 to 2 throughout the above. 

If as a result of these tests r sx — x ^, x S2 = x& 2 and c x + c 2 < y 
the difference between the settling times is sufficiently accurate 
for use in Rules 1 or 4, and the function c sx = x S2 — x si . Other¬ 
wise c sx — 0. (y is a constant chosen by considering the stability 
of the x\ and x x control loops.) 

After each time sweep the new values for the switching times 
are found by applying Rule 1. The other rules only apply 
occasionally, e.g. after a change in the desired output—Rule 2 
checks that the drive ratio obeys the condition found earlier; 
Rule 3 changes the mode or the drive direction when the initial 
guess for this has been wrong; Rule 4 governs the change from 
Mode 2 to Mode 1 or 3 whenever this is necessary; Rule 5 gives 
the switches in the plant drive whenever r' l9 x x or x 2 becomes zero. 
Rule 1 . Form new values for x x and r 2 by adding fu x c x to x x 


and /« 2 c 2 to r 2 . Also in Mode 1 form the new value for x\ by 
adding ^ x c sx to x x . ju x , ju 2 , yj x are constants sufficiently small for 
the pulsed control loops for t x , r 2 , x\ to be stable under all 
conditions (Figure 4). 

The speed of convergence of x x and x\ may be considerably 
increased by making the following additional corrections: in 
Mode 1, whenever a change is made in r' x , also add ^ x Cs X +^nCi 
+ Vzi c 2 to T i and 9Ti c i + 9 ? 2 a c 2 to r \ where rj sx etc. are constants 
(positive or negative) chosen by experiment to maximize the 
speed of convergence of x x and x\. 

Rule 2. In Mode 1, if in the plant U 2 I U x — sgn a 2X [and in 
the previous time sweep in the model \d 2 (r a2 )| > £, change to 
Mode 2 and set x x = ii x m^ (0)]. 

Rule 3. If in the previous time sweep d x (x ax ) > e (see above), 
set x x — x\ before applying Rule 1 and also, if in Mode 1 change 
to Mode 2, or if already in Mode 2 switch the plant drive U x . 

Rule 4. In Mode 2, if c si > s' changeto Mode 1 before apply¬ 
ing Rule 1 (s' is a small positive constant chosen by experi¬ 
ment). Also if the plant drive ratio U 2 /U x = sgn # 21 correct this 
by switching the plant drive U x . On the other hand, if already 
U 2 IU X — — sgn a 2X set x\ = r x before applying Rule 1 and then 
set r x = fjL x m x (0). 

Similarly if c S2 (= — c sx ) > c' change to Mode 3 and apply 
the above logic with the suffices 1 and 2 interchanged. 

Rule 5. After applying Rule 1, 

00 if in Mode 1, r' x < 0 switch the plant drive U x and change to 

Mode 2; 

(//) if in Mode 1, r x < 0 change to Mode 2 and set x x = ju x fn x (0); 

(iii) if in Mode 2, r x < 0 switch the plant drive U x and set 

r x =^ x /7? x (0); 

(iv) if in Mode 1 or 2, r 2 < 0 switch the plant drive U 2 and set 

r 2 = ju 2 m 2 (0). 

With an accurate model and input calculator, the accuracy 
of the control depends on the speed with which the searching 
process is carried out. This means that the model time scale 
should be about one thousandth of real time and that the length 
and number of time sweeps required should be minimized. 
Much of the complexity in the rules is due to this aim of using 
every piece of information available when computing new 
switching times, etc. Also a particular time sweep is ended as 
soon as all possible information has been obtained, as suggested 
by Chestnut 2 , by making the time sweep end at x r where: 



Figure 4 
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N°te. T r is the larger of r n and z r2 . To ensure the stability 
of the iterative process, the model drive u x (r) - 0 for r > v ri 
and u 2 (r) - 0 for r > r r2 . 1 

In practice, the optimum trajectory found in advance by 
the model and the actual.trajectory followed by the plant will 
differ slightly due to noise and other inaccuracies. The resulting 
recomputing of the optimum trajectory by the model could 
often cause the plant drive to be switched several times with 
consequent increase in power consumption and relay wear. 
For this reason the model only computes a new trajectory when 
it has moved, significantly away from the optimum one; the 
small quantities & and e in the rules are chosen accordingly. 
These rules give an on-off system which inevitably hunts when 
the plant output reaches the desired output, resulting in unneces¬ 
sary power dissipation. In general, providing the frequency of 
hunting is several octaves higher than the highest frequency 
which the plant is required to follow, the hunting may be 
smoothed out without loss of performance; and provided this 
is so the hunting in this system may be confined completely to 
the model by making the following changes in the controller: 

(1) Replace the power relays for the plant by low-power 
relays followed by low-pass filters and power amplifiers. Similar 
low-pass filters must of course also be included in the fast model. 

( 2 ) If m 1 ( 0 ) + m 2 ( 0 ) < 3 e always go to Mode 2 and modify 
Rule 5 (///) and (iv) to read as follows: 

If in Mode 2 , < e r switch the plant drive U% and set = 6 

where / = 1 , 2 . 6 is a constant chosen so that the model makes 
about five time sweeps between each switch of one plant relay. 

his last system has the advantage over many on-off systems 
that it will operate satisfactorily with ramp desired outputs. 
Under these conditions the above rules cause the square wave 
output of the plant relay to have a mark to space ratio appro¬ 
priate for following the desired ramp output. 



Figure 5 shows the projections on the output phase plane of 
a typical trajectory when the desired output Z is zero Figure 6 
shows the output, output rate and drive for the same trajectory 
plotted against time. 

Conclusions 

This paper presents a practical method for realizing the 
optimum controller for the two-variable system described by 
the differential equation x + A x= u, where the terms a,. 
and a a of the matrix A have the same sign. The controller is an 


on-off one using predicted changeover, and it is shown how 
the problem of hunting can be solved. Although the system 
described here has been developed for determinate but varying 
desired outputs, it is made applicable to random desired outputs 
by substituting an input predictor for the input calculator. 



As this system solves the Pontryagin equations in under fifty 
iterations it has considerable advantages over other iterative 
methods of solution, e.g. hill climbing, which may require 
several hundred iterations. It is proposed to extend this method 
to cover the more difficult case of a 12 and a 21 having opposite 
signs. The optimum controller in this case will require more 
than three switches; so the advantages of such a complex 
optimum controller will be weighed against the simplicity of 
a non-optimum controller based on the present method, by 
comparing back-traced optimum trajectories with non-optimum 
three-switch trajectories. 

Owing to the flexibility and logical nature of the rules 
governing this method, it is hoped that this type of controller 
will be applicable to a wide variety of two-variable plants and 
also that the logic may be extended to cover three, and eventually 
more, variable systems. 
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Summary 

A method of predicting a non-stationary process is described. It is 
shown that, if the sample functions of the process are expanded in 
terms of the eigenfunctions of the Loeve and Karhunen integral equa¬ 
tion, the integrated mean square error is a minimum with respect to 
variations in the eigenfunctions. Furthermore, this minimum error is 
equal to the sum of the omitted eigenvalues. By means of this expan¬ 
sion, the prediction problem is reduced to that of determining a finite 
number of coefficients which constitute a set of independent random 
variables. These coefficients are specified by applying a minimum 
mean square criterion to a suitable non-linear or linear function of the 
known parts of the sample function to be extrapolated. 

The method is also described in terms of discrete or sampled pro¬ 
cesses and is found to depend on an interesting theorem in matrix 
algebra. 

Finally, the method is applied to the problem of estimating the 
electricity demand in a large area several hours in advance and some 
results are given. 

Sommaire 

Le rapport decrit une methode de prediction d’un processus non 
stationnaire. II montre que si la fonction echantillonnee caracterisant 
le processus est exprimee dans les termes de la fonction propre de 
l’equation integrate de Loeve-Karhunen, le carre moyen de l’erreur 
est minimum par rapport aux variations de la fonction propre. L’erreur 
minimale est 6gale a la somme des valeurs propres omises. Ce deve- 
loppement reduit le probleme de prediction k la determination d’un 
nombre fini de coefficients qui constituent un ensemble de variables 
aleatoires inddpendantes; ces coefficients sont determines par l’appli- 
cation du critere du carre moyen minimum k une fonction appropriee, 
lindaire ou non,, des termes connus de la fonction echantillonnee qui 
est a extrapoler. 

La methode est decrite dans le cas d’un processus echantil- 
lonne; il ressort qu’elle est basee sur un theoreme tres interessant de 
l’algebre matricielle. 

Pour finir, cette methode est appliquee au probleme de la prevision 
plusieurs heures k l’avance de la consommation d’un grand reseau 
electrique et quelques resultats de Fapplication de cette methode sont 
donnds. 

Zusammenfassimg 

Der Aufsatz beschreibt ein Verfahren zur Vorhersage eines nicht- 
stationaren Prozesses und zeigt: Werden die Abtastfunktionen eines 
solchen Prozesses als Eigenfunktionen der Loeve-Karhunenschen 
Integralgleichung entwickelt, so ist der quadratische Mittelwert des 
Fehlers in bezug auf Anderungen der Eigenfunktionen ein Minimum. 
AuBerdem ist dieser minimale Fehler gleich der Summe der unter- 
driickten Eigenwerte. Diese Entwicklung reduziert das Vorhersage- 
problem auf die Bestimmung einer endlichen Anzahl von Koeffizien- 
ten, die eine Folge unabhangiger statistischer Variablen bilden. Diese 
Koeffizienten lassen sich naher bestimmen, wenn man ein Kriterium 
des kleinsten quadratischen Mittelwertes auf eine geeignete nichtlineare 
Oder lineare Funktion der bekannten Teile der getasteten Funktion 
zur Extrapolation anwendet. 

Bei der durchgeftihrten Erweiterung der Methode auf diskrete oder 
getastete Prozesse stellt sich heraus, daB sie auf einem interessanten 
Theorem der Matrizenrechnung beruht. 


Schliefilich wird die Methode dazu beniitzt, mehrere Stunden im 
voraus den Elektrizitatsbedarf in einem groBen Gebiet abzuschatzen; 
einige Ergebnisse sind dargestellt. 


Introduction 

The theory of Wiener 1 , in its original form, is concerned with 
the prediction and smoothing of stationary processes by linear 
filters; Wiener’s method has, however, been extended by 
Booton 2 to the processing of non-stationary time-series by 
time-variant devices and by Zadeh 3 to non-linear filtering. 
However, Wiener’s theory and its extensions do not take 
account of any constraints to which the process may be subject. 
For example, the process might contain natural ‘modes’, a 
knowledge of which would certainly facilitate its prediction. 

In the following, it is shown that the characteristic modes 
of a process may be defined in an unambiguous and optimum 
manner. Methods of predicting the process from a knowledge 
of its modes are described. 


The Generalized Wiener Predictor 


In the generalized form, the prediction problem is that of 
synthesizing a time-variant linear filter whose output, y (?) at 
any time ?, is an estimate of the input at the later time ? + oc. 
More precisely, if x (?) is a typical member of a non-stationary 
statistical process, the filter is required to minimize the mean 
square error [e 2 (?)] between the output y (?) and the input 
x (? + oc) at time ? + oc. If e (?) is the error at time ?, then 

e(t) = y(t)-x(t + a) 

The mean square error [e 2 (?)] is obtained by squaring s (?) and 
averaging over the group of inputs and outputs 

[e 2 (*)] = [ y 2 (0] - [2 y (0 x(t + a)] + lx 2 (t + a)] (1) 


(Here the square brackets denote group averaging.) 

Let g (?, r) denote the output of the filter at time ? due to a 
unit impulse input (delta function) at time r; then the output 
y (?) due to the input x (?) is given by 


y{ 0 = 


't 

0 


g(t,x)x(x)dx 


( 2 ) 


Combining eqns (1) and (2) gives 


L 2 (0] = 


g (f, T) g (t, t') R (t, t') drdr' 


-2 g(t,x)R(t + a,r)dx + R(t + a, t + a) ( 3 ) 

' 0 
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where R (r, t') denotes the autocorrelation function of the 
process 

R (r, t') = [x (t) x (t')] (4) 

If the response function g (/, r) is varied by dg ( t , r) then (to 
a first order) the corresponding variation in [e 2 ( t )] is 

[a e 2 (0]=2f f %(f,T){P 

Jo (Jo 

If g (; t , r) is the response function of the optimum filter which 
minimizes [s 2 (/)], then the variation [S e 2 (/)] vanishes for all 
dg (t, t) ^tid hence 


R t') g(t 9 t') dz' - R(t + a, z) > dz 


some sense, the characteristic modes of the process. A simple 
way of specifying the first mode is to seek a function faCt), a 
scaling factor Xfi and a set of coefficients a ml such that 
h* a mi ^i(0 approximates, in a least-squares sense, to the 
sample functions x m (t) over a specified time interval (0, T). The 
rath error takes the form 

« m (0 = X m ( 0 - 2 ffl rai ^> 1 (0 

The mean square error averaged over time and over the group is: 

1 M C T 

E K(0-4«m>i(0} 2 df (7) 

1VI 1 m = 1 J 0 


& (t, z') g (f, z') dr' = JR (t + a, z) 0 < z < t (5) 
J o 

The solution g (t, z) of this modified Wiener-Hopf integral 
equation is the response of the optimum Wiener filter and the 
resulting mean square error is given by 

[s 2 0)]min = £ (t + a, t + a) - f g (f, T) R (t +a, x) dx (6) 

The first term on the right is the mean power [x 2 (t + oc)], 
flowing at the time / + oc, and the second term represents the 
maximum difference between the mean power and the mean 
square error that it is possible to obtain by linear filtering. 

The response function g (t, r) of the Wiener filter is specified 
uniquely [cf. eqn (5)] by the autocorrelation function R (r, r') 
of the process. The applicability of the predictor depends, 
therefore, on the possibility of constructing a representative and 
realistic autocorrelation function which, in practice, must be 
derived from a finite number of the sample functions of the 
process. Suppose, for example, that the autocorrelation function 
R (t, t') is formed from the M sample functions x m (t\ (ra = 1, 
2, 3, ... M) then 

i M 

R ( X ’ T ’) = M £ X m(0*m(O 
IV1 m= 1 

The sample functions x m (t) are determined ideally by performing 
M sets of measurement under identical conditions. However, in 
many practical situations the number of sample functions it is 
possible to measure under almost identical conditions is severely 
limited by uncontrollable variations in one or more of the para¬ 
meters of the process. For example, if the functions x m (t) were 
daily curves of electricity demand, then the number which it is 
practically useful to include, in forming the autocorrelation 
function, is limited by the seasonal trend of the load and by the 
economic growth in demand. However, for a large class of 
stochastic processes, the sample functions are expressible in 
terms of a small number of ‘characteristic modes’ which arise 
from the physical nature of the source of the particular process. 
Under such conditions the inclusion of a large number of sample 
functions in calculating the autocorrelation function is neither 
necessary nor desirable. Furthermore it is the ‘modes’ rather 
than the correlation function alone which characterizes the pro¬ 
cess and the Wiener predictor is no longer appropriate. 


The constants X-ft a mi and the function ^(t) may be chosen 
such that this error E is a minimum. This minimum is attained 
when the variation in E due to first order variations in X x % a m 
and <^ 1 ( t) vanishes, i.e. when 

T i rr 

<l>i(f)x m (t)dt=l\a m \ (j>\ (t)dt m = l,2,...,M 
Jo Jo 

M M 

X! (0 ~ X; (f) 0 <C t < T 

m = 1 m = 1 


The function (p^t) and the vector a mi may, without loss of 
generality, be normalized such that 

k r 

<£i(Y)df=l 

0 
M 

E < = M 

m= 1 

The minimum conditions then take the form 

Ct 

a " l " Ar *J 0 ( ^ )i ^ Xm ^ dt 


i M 

E a mi xjt) 


m—1 


Elimination of a ml between these two relations yields 

1 


M m=lJo 


E fW x, n (%) drx m (0 = A 1 (-/. 1 (r) 


or 


R(t,'c)(j> 1 (z)dz=X 1 (l> 1 (t) 0 <t<T 


( 8 ) 


(9) 


where R(t, r) is the correlation function formed from the group 
of the M sample functions 

j M 

R ^ = M £ x m(f)x m {T) ( 10 ) 

1V1 m = 1 

Combining eqns (7) and (9) gives for the minimum mean square 
error 


E- 


f r 


R (t, t) dr - A 


( 11 ) 


Derivation of the Characteristic Modes 

Suppose that m= 1, 2, 3, .. M, are M sample func¬ 
tions of a stochastic process and that it is required to define, in 


Eqn (9) shows that (i) is an eigenfunction of the modified 
Wiener-Hopf integral equation. However, eqn (11) indicates 
that the error is a minimum when 2, is the greatest or dominant 
eigenvalue and hence ^> 1 (t) is the dominant eigenfunction. The 
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second mode may be, similarly, specified by requiring that 
a m2 (j) z (t ) be the best mean-square approximation to the 
error e m (t) and it follows that X 2 is the second largest eigenvalue 
and cj) 2 (t) is the corresponding eigenfunction. Continuing in this 
way an expansion for x m (t) is obtained and this takes the form 

= 1 (f) ^ ^2 a m2 < t > 2 (0"b ^3^m3^3 (0"b 


where (j> k (t) and X k satisfy the eigenfunction equation 

0 <t<T 

R (t, t) 4> k (t) dr = A k cj) k (t) = 1 2s 3 ,... 

jo 

and where the coefficients a mk are given by 


A mk ' 




~ £• 


<i>k (0 x m (f) 


( 12 ) 


(13) 


An important property of the expansion of x m (t ) is that, if the 
series is terminated after K terms, the time integrated mean 
square error ET takes the form 


ET— 


The term 


R(x,x')dx’~ Yu 4 

0 fc= 1 


jR(t,t') dr 


(18) 


is equal to the average energy contained in the process over the 
time interval (0 ,T). By virtue of eqns (16) and (14) this energy 
takes the form 

m T oo 

jR(T,t)dT= Y 4 ( 19 ) 

Jo k-1 

Combining eqns (18) and (19) gives 


It follows from the general theory 4 of the eigenvalue equation, 
that the functions (j) 1c (t) are orthogonal and as they are nor¬ 
malized form an orthonormal set 

fr fl if k' = k 

J q <4 CO 4 4- (0 dT = s kk -= |o ^ k > # k ( 14 ) 

The constant a mk may be regarded as the value which a random 
variable a k takes for the rath number of the group. The cor¬ 
relation between the random variables a k and a k < then takes 
the form 

j M 

{. a k a k'h = ~M Y a mk a mk' 


From eqn (13) it then follows that 


1 


M 


lvl m — 1 


[*T 

J 0 


„(0 xjx') 4> k (t) 4> k - CO dxdx 


= (440 




t r r 

0 J 0 


R (t, x') 4> k (t) (j) k , (x') dxdx' 


=(440 -i 4 


4>k<d4>k'(?) ii: 


— d kk' l - ®- [«***'] 4. 


(15) 


Thus the random variables a k possess unit mean square values 
and zero cross correlations. It is evident that the mode func¬ 
tions 4> k (r) are identical to those introduced by Loeve 5 and 
Karhunen 6 in deriving an expansion with uncorrelated coeffi¬ 
cients; for this reason eqn (12) might be termed the Loeve- 
Karhunen integral equation. 

The autocorrelation function R( r,r') may also be expanded 
in terms of the characteristic modes of the process 

00 

R(x,x')= Y 40fc(*)<£*CO ( 16 > 

k=l 

It may also be shown that all the eigenvalues X k are either 
positive or zero 

A fc >0 (17) 


ET = t 4 (20) 

k = K+ 1 

Thus the integrated square error ET is equal to the sum of the 
omitted eigenvalues and the number of terms required for a 
specified accuracy is readily obtainable from eqn (18) or (20). 
The eigenvalue X k , being essentially non-negative, may be inter¬ 
preted as the energy associated with the k th mode of the 
process. 

The impulse response of the Wiener predictor may be de¬ 
scribed in terms of the modes Suppose, for example, that 

the predictor is required to estimate an input sample function 
at the time T = t + oc from a knowledge of the input over the 
time interval (0,r o ) where T 0 < T. The impulse response satisfies 
the relation 

T ° R (t, t') g (T 0 ,0 dr' = R (T, t) 0 < t < T 0 

J o 

If i?(r,r') is expanded in the form of eqn (16) the integral 
equation takes the form 


where 


l4^*(0=£4<Mm*(0 o<t<t 0 

k k 


gk = 


'T 0 

0 


g(T 0 ,x) cj) k (t) dx 


( 21 ) 


Multiplication by <f> k '(r) and integration over the range (0,r 0 ) 
yields 

YAk-*k'gk'=YAkk’h4AT) (22) 


where 



<4(0 <MO dT 


(23) 


As the function given by the series 

ZgACO 

k 

must vanish in the range T 0 < r < T it follows that 


where 


Y B kk'g k '=o 

Ir' 



<4 CO <4-CO dx 


To 


(24) 

(25) 
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The matrices A and B with elements A kk , and B lck , respectively 
are idem-potent in that they satisfy the relations 

A 2 = A, B 2 = B (26) 

Furthermore, as the mode functions are orthonormal over the 
whole range (0, T) it follows that 

A + B = I 

ab=ba=o ( 2? ) 

where / denotes the unit matrix. In terms of the mode functions 
the mean square prediction error of eqn (3) takes the form 


method of prediction automatically treats the sample function 
x(t) as a combination of its characteristic modes whereas the 
Wiener predictor imposes no such constraints on x(t) treating 
it quite generally. There are several methods of specifying the 
coefficients c k depending on the error criterion adopted Two 
possible methods are described below. 

Method 1 A particularly simple procedure is to choose the 
coefficients c k such that eqn (30) is the best mean square approxi¬ 
mation to x(t ) over the time interval (0,7k) for which x(t) is 
known. 

If e(t) denotes the expansion error then 


[s 2 (T 0 )] == £ X k {g k - 4> k (T)} 2 (28) 

k = 1 v ' 

This error is minimized when g k satisfies eqns (22) and (24) 
i.e. when v 

£ A kk'\'g k . = Y, A kk .X k .<j) k , (T) 

k f k > 

^ (29) 

^j^kk’Sk ,= 0 
k■ 

It follows from eqns (26) and (27) that the sum of the ranks of 
the two matrices A and B is equal to the rank of the unit matrix/ 
Consequently in eqn (29) there are exactly as many independent 
equations as there are unknowns g k and the solution is unique. 
These simultaneous equations are completely equivalent to the 
Wiener-Hopf relation (5). 

The contribution of the term in g* to the prediction error is 
proportional to the energy or eigenvalue 4 [this is demonstrated 
by eqn (28)]. It follows then that any error in the value of g, 
does not appreciably increase the prediction error provided 
that the corresponding eigenvalue 4 is sufficiently small. In 
particular, those coefficients g k corresponding to negligible 
eigenvalues 4 may be neglected, i.e. modes with negligible mean 
energies may be ignored. 

Prediction of a Process by its Characteristic Modes 

The prediction problem is essentially that of estimating the 
values of a sample function x(t) of a non-stationary process in 
the time interval (T 0 ,T) from a knowledge of x(t) in the range 
V4 ■* oh 

It has been established, eqn (18), that if the process is ex¬ 
panded to K terms as a combination of its characteristic modes 
then the integrated mean square error ET takes the form 

£r= (4(r, x )dr'-;£ 4 

dO fc=l 

From this criterion the number K of terms which achieve a 
specified accuracy may be derived. To this order accuracy the 
sample function x(j) may be expressed in the form 

K 

*(0=£c*&(0 0 <t<T (30) 

where c k are a set of constants. This expansion is valid over the 
whole time interval (0,2") including the part of the range in 
which x(f) is to be predicted, provided that the coefficients c,. 
are specified appropriately. The prediction problem, therefore, 
reduces to that of determining the set of constants c k . This 


e(t)=x(t)- £ c k <j> k {t ) 

k= 1 


The integrated square error D is given by 


and hence 


£= °s 2 (t)dt 

J o 

'-L'fa-i*'®} 2 * 


The error D is a minimum when 


-=0 k = 


and hence when 


* f r ° fr 0 

t Z, C *' L &<»&'(*)dx= x(t)<4( z)dr k = l,2, ...,K 

n -1 J 0 J Q 

(32) 

This set of simultaneous linear equations determines the coeffi¬ 
cients c k uniquely. The prediction of x(j) is effected by applying 
eqn (30) over the time interval ( T 0 , T). 

Method 2—An alternative method of specifying the coeffi¬ 
cients c k is to require that they approximate to the true coeffi¬ 
cients in a least-squares sense. The true coefficients, as is evident 
irom eqn (8), take the form d k where 

d k = x(t)(j) k (t)dt ( 33 ) 

J 0 

On the other hand, the constants c k depend only on the values 
of x(t) in the range (0, T 0 ) and it has been shown by Volterra 7 
that the most general operation of this type on a function x(t) 
must take the form 

* T ° [To [To 

c k= fk(r)x (t) dr + h k (t, t') x (t) x (t') dtdi' + ... 


If e jc denotes the error between c k and d k then, using eqn (33), 
the following relation is obtained for the mean square error [s 2 k ] 

[t Pr 

[ £ fe] = R i x A')(t> k (x)4> k (x') drdr' 

J 0 J 0 


-2 ^ [ c k x (t)] <p k (t) dr + \_c k ~\ 
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The functions f ]c ( r), h k (x, x'), etc. which occur in eqn (34) may 
be chosen so as to minimize [e k 2 ]. To effect this minimization 
c k is substituted for in eqn (35) and gives 

I Eqn (36) I 


The error [fi 7c 2 ] is a minimum when its first variation with respect 
to variations in / 7c (r), h k (r',x), etc. vanishes, i.e. when 


Eqn (37) 


t 


The functions f k (r), h k (r,r") etc. which satisfy eqn (37) define 
an optimum non-linear predictor for the process x(t). The 
determination of this predictor requires, however, a knowledge 
of the higher order covariances [x(r) x(t') x(t")] etc. If this 
treatment is restricted here to the case of linear predictors the 
functions h k (r,r f ) etc. may be neglected and eqn (37) takes the 
comparatively simple form 

R(x,x')f k (x‘)d 0<T<r o (38) 

J 0 

For each value of k in the range 1 < k < K, eqn (38) defines a 
function f k (r) which, in turn, determines the weighting coeffi¬ 
cient c k of the /cth mode according to the relation 


c k z=z 


x(-c)f k (i)dx 


(39) 


J o 

The predicted values of x(t) are obtained from eqn (30) as be¬ 
fore. A matrix formulation of the prediction method which 
lends itself directly to digital computation is described in the 
Appendix. 


Application to the Problem of Load Estimation 

The problem considered here is that of predicting the elec¬ 
tricity demand in a large area, up to several hours ahead. Predic¬ 
tions of load are required for the purposes of ordering generating 
plant, loading of plant and checking the security of the supply 
network in advance. If automatic control is to be applied to a 
supply system then the prediction of load by digital computation 
becomes a necessity. 

For the purpose of prediction, each daily load curve may be 
divided into part-day periods of several hours’ duration, the 


periods including the interval (T 0 , T) over which the prediction 
is required together with an immediately preceding interval 
(0,r 0 ); the period as a whole is then coextensive with the 
period (0,T). The load during each of these periods depends 
on many factors including meteorological parameters and the 
response of the consumer to television and radio programmes, 
etc.; of these factors the meteorological ones have the greatest 
influence. It is plausible, therefore, to express the load x mn on 
the mth period at the nth instant of time in the form 

* m „ = oc„ +/i (TJ /?„ +f 2 (L m ) y H +/ 3 (WJ 8„+... (40) 

where A(T m ), f 2 (L m \ are functions of temperature T m , 

light intensity L m , wind velocity W m respectively. The quantity 
oc n represents the base load and the factors (3 n , y n , S n , allow for 
the varying importance of weather parameters with time of 
day. According to eqn (40) each load vector is linearly dependent 
on the vectors oc, /3, y, d , ..., however, if the load is expanded 
to K terms as a combination of its characteristics modes, the 
quantity x mn takes the form 

K 

X m „= £ c mk <j) kn (41) 

k — 1 

It has been shown previously that the mode vectors <j> k minimize 
the error of the expansion and it is evident that if the mode 
vectors were replaced by K linearly independent combinations 
then the error is unchanged. Thus the K dimensional manifold 
formed by the mode vectors and their linear combinations gives 
a smaller error than any other K dimensional manifold. It is 
plausible, therefore, to regard the vectors oc, /3, y, S, ... of 
eqn (40) as belonging to the manifold of mode vectors. It is 
concluded that the modes describe the basic trends of the load 
under average weather conditions for the period of the records 
whereas the weighting coefficients c mk depend on the meteor¬ 
ological parameters relevant to the mth period. 

Results 

The method of characteristic modes has been used to predict 
loads up to 8 h ahead in a large area with a peak demand of 
some 5,000 MW. The weighting coefficients were calculated 
by the procedure described as Method 1. Computer results, 


Eqn (36): 


[4]=4 
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J 0 


hf k (x)<t> k (x)&x+ 

To 


To fr 0 


o J o 


R(x,x')f k (x)f k (x')dx&x' 
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'To CTo 
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p Tq TQ J*Q 
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'To 


[x( t) x( t') x( t") x( t'")] h k (z, t') h k (t" 5 t") didi'di'Mi:"' +... 


[x (t) 2c (t ; ) x (t")] h k (i', %") <i> k {x) 


(36) 


Eqn (37): 

To 


0 


R (x, X’) f k (z') dr' - (t) + 


To 


[x (t) x (t*) x (t")] f k (z) dz - 
etc. 


T 0 

0 J 
T 


To 


[x (t) x (t') x (t")] h k (t', t") dt dz" +... = 0 

To f To 


[x (t) x (t') x( t")] 4> k {z)dz+ j j [x (t) x (t')x(t") x CO] h k (x,z'")dzdx"' + ...— 0(37) 

Jo Jo 
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obtained to date, give predictions for 30 weekdays covering six 
aitterent periods each day. The characteristic modes were cal- 
7* ate “> usl ng the data of up to 20 immediately preceding week- 
ys. The calculations were performed on an IBM 7090 com- 
PU er and about 1 min of machine time was used in the cal¬ 
culation of each predicted curve. It was found that use of five or 
six eigenvectors gave the best results, depending on the length 
ot the prediction and that the R.M.S. error was about 3 per cent 
tor predictions of 3 to 4 h ahead. 

•it < T UrVe f . com P arin g predicted load with actual load are 
i lustrated m Figures 1-8 for eight typical weekdays. Each of 
predicted curves was calculated 30 min before its commence¬ 


ment. The most important predictions, for the purpose of 
ordering generating plant in advance, are those which cover the 
breakfast peak at about 08.30 hours and the evening peak at 
about 18.00 hours. Results obtained for the breakfast peak com¬ 
pare favourably with predictions made at control centres using 
meteorological data, whereas the evening predictions were found 
to be less accurate on average. 

The problem of estimating the load from its characteristic 
modes is being further investigated. In particular the procedure 
described as Method 2 will be tested although no results are 
available at the present time. In addition the problem of deriving 
confidence limits for the predictions is being studied. 



Figure 1. Predicted and actual load on Monday 27th November Figure 7 . , ,, , „ 

rigure 2. Predicted and actual load on Tuesday 28th November 



FltureS. WeA^y 29,h November ^,4. .. ... 


52 








A METHOD OF PREDICTION FOR NON-STATIONARY PROCESSES AND ITS APPLICATION TO THE PROBLEM OF LOAD ESTIMATION 



Figure 5. Predicted and actual load on Monday 4th December 
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Figure 6. Predicted and actual load on Wednesday 6th December 
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Figure 7. Predicted and actual load on Thursday 7th December Figure 8. Predicted and actual load on Thursday 14th December 

Appendix 

Matrix Formulation 

For the purposes of digital computation it is more con¬ 
venient to treat the process x(t) as a discrete function of time 
rather than a continuous one. Suppose, then, that the process is 
sampled at instants of time separated by equal intervals, the 
sampling frequency being sufficiently high compared with the 
reciprocal bandwidth, and let x mn denote the value of the rath 
member of the group at nth. instant of time. The elements x mn 
form a rectangular matrix X , 

X=(x mn ) 

If there are M members of the group and N sampling instants 
then X has M rows and N columns. Corresponding to the auto¬ 


correlation R(r,r ) there exists a covariance matrix R which is 
defined by 

R=±-X'X (42) 

M 


where X' denotes the transpose of the matrix X. 

The expansion of the process in terms of its characteristic 
modes is equivalent to expressing the elements of the matrix X 
in the form 


X mn— Yj a mk^k4 > k 


Denoting the errors by e mn , i.e. writing 


E mn = X m „- £ a mk X$(j) k 
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and choosing a mk and (j) kn such that the sum of the squares of 
s mn is a minimum gives 

N 

^k a mk~ X X mn4 > kn 

n= 1 

M ( 44 ) 

MlU kn = X a mk X mn 
m~ 1 


This set of equations is equivalent to eqn (8). Letting P denote 
the matrix with elements a m!ci Q the matrix with elements 4 ^Jcn 
and A the diagonal matrix with elements A k then eqn (43) takes 
the form 


X=PA±Q 

and eqns (44) become 

PA* = XQ' 
MA*Q=P'X 


(45) 

(46) 


Eliminating P and Q in turn from eqn (46) gives the result 


RQ'=AQ' 
SP = PA 


(47) 


where S is the matrix given by 


where cj) kn are the elements of Q which is specified by eqn (47). 
Method 1 determines the coefficients c k according to the re¬ 
lations 

Z cA Z Kfam } = Z &»*» k.= l,2,K (49) 

k'= 1 U= 1 J n= 1 

These are K simultaneous equations for the K unknowns c k . 
In the linear form of Method 2 the coefficients c k are written in 
the form 

c k =lx n f kn k=l,2, ...,K (50) 

n=l 

Corresponding to eqn (50) the constants f kn are specified by the 
relations 

N ' n = 12 3 N' 

Z Rnn’fkn-Wkn ^ 

The sum of eigenvalues of the matrix R is equal to the sum of 
its diagonal elements or its trace; the error criterion analogous 
to eqn (18), therefore takes the form 

K oo 

NE= trace!?- Z Z 4 ( 52 ) 

k=l k=K+ 1 


5 = 


M 


XX' 


The author wishes to thank the Director of the Central Elec¬ 
tricity Research Laboratory for permission to publish this paper. 


The first relation of (47) is the matrix form of the integral 
eqn (12). 

It has been established that for any rectangular matrix X the 
best approximation of the form (45) is attained when P and Q 
satisfy the eigenvalue eqns (47). 

In its matrix form, the prediction problem is that of estimat¬ 
ing the values of a vector in the range n = N' + 1, n' 4- 2, 

Ngiven the values of x n in the range n = 1, 2, 3, ..., N'. 
Corresponding to eqn (30), x n is expressed in the form 

*»= Z c k<t>kn (48) 

k — X 
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Summary 

The purpose of this paper is to investigate the optimum design of 
predictor control systems subject to statistical or deterministic inputs, 
under appropriate constraints on the control signal. The control 
signal constraints treated in this paper are: (a) the magnitude of the 
control signal equal to a constant (this is the bang-bang control 
problem), and (Z>) the magnitude of the control signal less than or equal 
to some constant. 

The basic optimization problem is viewed, in very general terms, 
as a problem in decision theory. Based on available information, the 
decision must be as to which of the allowable control signals should 
be applied to the plant in order to achieve the optimum performance. 
A sum of squared error criterion is used to define the optimum in 
order to reduce the problem to a problem in linear decision theory. A 
piecewise-linear control boundary is found which can be used to 
generate the desired control signal. The realization of this control 
boundary is shown to be straightforward and inexpensive. This 
technique can be applied to linear time invariant plants of arbitrary 
order. Experimental results verifying the techniques which are devel¬ 
oped are presented. Included in these is the control of the step response 
of a system with a saturation-limited unstable second order plant in 
the presence of noise. 

Sommaire 

Le but de ce rapport est d’etudier le mode de realisation optimale 
d’un systeme de prediction sounds a des variations pred6terminees ou 
statistiques et compte tenu de certaines contraintes imposees au signal 
de commande. Les contraintes prises en consideration dans ce rap¬ 
port sont les suivantes: (<z) l’amplitude du signal de commande est 
6gale a une constante (‘bang-bang control problem’) et ( b ) Tampli- 
tude de ce signal est egale ou inferieure a une valeur prescrite. 

Le principe de ce probleme d’optimalisation est examine tout 
d’abord sous sa forme la plus generate, comme un probleme de la 
th6orie des decisions. En utilisant les informations disponibles il 
s’agit de choisir la commande a appliquer au systeme en vue de r6a- 
User une performance optimale; le critere de la somme des carres 
moyens des erreurs est utilise pour ramener ce probleme a celui 
traite par la theorie de la decision lineaire. Une linearisation des 
contraintes peut etre trouvee permettant de produire le signal desire. 
Le rapport indique que la realisation d’une telle contrainte est efficace 
et relativement peu couteuse. Cette technique peut etre utilisee pour 
des installations lineaires invariantes dans le temps d’un ordre quel- 
conque. Des resultats experimental verifiant le bien-fonde de cette 
technique, sont presente. Parmi ces resultats figurent le reglage de la 
reponse indicielle d’un systeme avec une installation instable du 2me 
ordre presentant des phenomenes de saturation en presence du bruit. 

Zusammenfassung 

Der Aufsatz untersucht den optimalen Entwurf eines Regelsystems 
mit Vorhersageeigenschaften bei regellosen oder deterministischen 
EingangsgroBen, wobei das Regelsignal entsprechenden Beschran- 
kungen (Bedingungen) unterliegt. Die hier behandelten Beschrankun- 
gen des Regelsignals sind: a) der Betrag des Regelsignals ist konstant 
(Zweipunkt-Regelproblem) und b) der Betrag des Regelsignals ist 
kleiner oder gleich einer Konstanten. 

Das grundlegende Optimierungsproblem wird hier recht allge- 
mein als ein Problem der Entscheidungstheorie betrachtet. Auf Grund 


der zur Verfugung stehenden Information ist zu entscheiden, welches 
der zulassigen Regelsignale der Strecke zuzufiihren ist, um ein opti- 
males Verhalten zu bekommen. Damit man das Problem auf ein 
solches der linearen Entscheidungstheorie zuruckfuhren kann, wird 
das Optimum als Summe des quadratischen Fehlers definiert. Eine 
abschnittsweise lineare Grenze der RegelgroBe wird gefunden, die 
sich zur Erzeugung des gewiinschten Regelsignals verwenden laBt. 
Die Verwirklichung dieser Grenze des Regelsignals erweist sich als 
einfach und billig. Diese Methode laBt sich auf lineare, zeitunabhan- 
gige Strecken beliebiger Ordnung anwenden. Die Arbeit enthalt 
einige Versuchsergebnisse, die diese Methoden bestatigen; so z. B. 
bei vorhandenem Rauschsignal die Regelung der Sprungantwort 
eines Systems mit einer durch ein Sattigungsglied begrenzten *in- 
stabilen Regelstrecke zweiter Ordnung. 


Statement of the Problem 

The problem being considered is the design of a predictor con¬ 
trol system to achieve performance which is optimum in some 
sense 1 . Figure 1 is a block diagram of the system under con¬ 
sideration. 

In the design problem, the performance index is first chosen. 
The control computer which will optimize the system with 
respect to that performance index must then be found. In this 
problem, the plant is assumed to be fixed and known, and the 
control signal is constrained in some way. 

The control signal constraint of most interest is saturation. 
If m (r) is the control signal, this constraint is expressed as 

\m(t)\<M (1) 

Systems with control signals governed by the constraint 
\m(t)\ —M will be referred to as ‘bang-bang’ control systems. 

The selection of a performance index is a compromise 
between what might be considered to be truly optimum per¬ 
formance and a practical solution. A practical solution is charac¬ 
terized by a straightforward design procedure and a simple 
control computer. 


r(t) 

Control 

m(t) 

Linear i 

| c(t) 


computer 


plant 

1 



c(t)| 







Figure 1 


With this in mind, the error criterion chosen can best be 
explained in terms of a hypothetical system. This system has 
a piece wise-constant control signal (it can chan ge values only 
at times t ]c ) and is subjected to an input r{t ). It contains a 
control computer which employs a sub-optimal control law. 
This type of control law is sometimes referred to as sub-optimal 
since it does not take into account the fact that, in practice, 
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future decisions will be based on information not available 
at t = t 0 . Instead, it generates only the initial value of the 
control signal arrived at through the resulting optimization 
and continually repeats the process, c* (t) represents the output 
of the hypothetical system. The performance which is used to 
define the optimum is the minimization of 


N 


1= £ (r k ~c* k f 
1 


= £ 


k= 1 


( 2 ) 


with respect to c k * [the notation used here is/( 4 ) = f k ]. 

For deterministic inputs, r k — r k and optimum perform¬ 
ance corresponds to the minimization of the sum of squares of 
future errors of the hypothetical system. For stochastic inputs, 
minimization of 


N 

£ 0 k- 

U= 1 


C.i/f 


,p ] 


where P represents the history of r(f) 9 and c(t ) is equivalent to 
minimization of I with respect to cf* where 


yjpi 

For Gaussian processes, r k can be found as the output of a least- 
squares predictor. The actual system under consideration differs 
from the hypothetical system previously considered in that the 
actual control signal is not constrained to be piecewise constant. 

As a result of this differencejbetween the actual and hypo¬ 
thetical systems, neither r k nor e k will in general appear at any 
time in the actual system. The error criterion is none the less 
reasonable and will form the basis of the work to follow. 


Formulation of Problem 


where 


z k~ r k~ c k0 


( 7 ) 


Thus > z k is determined by the behaviour of the reference 
input [r(f)L and by the state of the system [this determines c k0 ]. 
Neither of these quantities can in any way be altered by the 
control computer. co k , on the other hand, represents the change 
in output due to allowable changes in the control signal. The 
allowable values of a> k are completely determined by the plant 
and the control signal constraints. Thus, if one writes 


'= ii ffi’- £ 

k=1 k=l k=1 

one notes that minimization of I with respect to c k * is entirely 
equivalent to minimization of I with respect to co k . 

The application of linear decision theory to the solution of 
the optimization problem at hand will now be demonstrated. 
Before proceeding further, it is necessary to introduce the 
following notation. 

{m\ = [m l9 m 2 , ..., mjg] represents the control sequence. 
This notation is used to represent a signal, m(t ), which has the 
value m 1 for 0 < t < t x , m 2 for t x < t < t 2 , ... etc. The various 
elements of this control sequence are appropriately constrained. 

I 5 ”) ~ (^o s i> • • ‘S’n-i} represents the switching sequence. 
The constraints on the 61ements of this sequence are related to 
the control signal constraints through eqn (3). 

{&>} = { a) l9 co 2 , ..., co^}. The constraints on the elements of 
the co sequence are related to the switching signal constraints 
through eqn (5). Corresponding to each control sequence there 
is an co sequence. 

{f} ~ { z i> z 2 > * • *j zjg], The elements of this sequence are de¬ 
termined by the reference input and by the state of the plant. 


c{t) may be expressed as the sum of two signals. The first of 
these represents the output which would exist if the control 
signal were cut off at t = t 0 . This is simply the solution of the 
homogenous differential equation describing the plant. This is 
referred to as c 0 (t). The second signal represents the change in 
output due to allowable changes in the control signal. This is 
referred to as co(t). To express co(t), it is advantageous to define 
a signal which represents the change in control signal at the 
various discrete times, t k9 when such a change is assumed per¬ 
missible. This signal will be referred to as the switching signal, 
Sji, and will have the property 


Sk=™ k+1 -m k (3) 

where m k = m(q). Thus m, c represents the control signal im¬ 
mediately before 4 and m k+1 represents the control signal im¬ 
mediately after. 4 thus represents the change in control signal 
occurring at 4 . 

The plant being considered is described by either its impulse 
response, g(f), or its step response, h(t). The initial value of the 
step response is assumed to be zero. 

Using the notation introduced above, one has 


where 


C k — 

(4) 

k- 1 


04= E K-jSj 

(5) 

i=o 

= r k~~ C k0 CO k = Z k — CO k 

( 6 ) 


Theorem 1 —The sum of squared error criterion being employed 
in eqn ( 2 ) is a minimum-distance criterion 2 . 

Proof -—It is desired to minimize 

/= £ 0 k-co k .) 2 

k= 1 

with respect to co k subject to the appropriate control signal con¬ 
straints If one considers an iV-dimensional space, a point, co, 
the elements of which are oj.j, may be associated with each 
sequence {«}. The finite number of such points associated with 
a bang-bang control signal constraint will be referred to as 
pattern points. Similarly, corresponding to each sequence of z’s, 
{zj, there exists a point, z, in iV-dimensional space. 

Geometrically, it may be said that the minimization of/with 
respect to co corresponds to choosing the co point nearest to z. 
Corresponding to this co sequence, the optimum control sequence 
can then be found from eqn ( 5 ) and eqn ( 3 ). 

Theorem /—Linear decision functions are optimum for prob¬ 
lems employing minimum-distance decision functions 2 . 

Proof —A linear decision function has the property that it 
divides measurement space into regions such that each pair of 
regions is separated by one and only one hyperplane. Measure¬ 
ment space is the JV-dimensional space in which the pattern 
points and the z points are plotted. 

Figure 2(a) illustrates a minimum-distance criterion. Three 
pattern points are Shown. A measurement Q is identified'with 
that pattern point which is closest to it in an Euclidean sense. 
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Consider Pattern Points 1 and 2 (PP1 and PP 2) and the hyper¬ 
plane (line in this two-dimensional case) B 12 which is the per¬ 
pendicular bisector of the line joining PP 1 and PP 2 [Figure 2 (&)]. 
Then the statement that a point Q is closer to PP 1 than to PP2 
is equivalent to the statement that the point lies on the 1 side 
of jB 12 . By constructing a hyperplane for every pair of pattern 
points, a linear decision function equivalent to the minimum- 
distance decision function is obtained. 

The region associated with each pattern point will be referred 
to as a pattern class. Thus, associated with PP 1 one has Pattern 
Class 1. 


is to be positive or negative is completely arbitrary, since multi¬ 
plication of eqn (8) by -1 changes the sign of d but does not 
change the hyperplane. 

Implementation of a Hyperplane 

Figure 3 indicates the simplicity of implementation inherent 
in linear decision functions. The distance, d , of a point, z, from 
the hyperplane in question is proportional to the current L The 
direction of current flow indicates the side of the hyperplane on 
which the point is located. The scale factor enables the use of 
realistic component values and signal levels. 




Figure 2. {a) Minimum-distance decision function; (b) linear decision 
function equivalent 


Representation of a Hyperplane im Normalized Form 
Consider a hyperplane given by 

£a £ X; + <x 0 = 0 ( 8 ) 

£= 1 

where 

i>?=i w 

i=l 

Then the oc { , 1 < i < K are the direction cosines of the 
hyperplane, and oc 0 is its distance from the origin. The distance, 
d, of a point P from this hyperplane is simply given by sub¬ 
stituting the coordinates of the point into the normalized equa¬ 
tion of the hyperplane; 

d=Yi a i P i +a o ( 10 ) 

i=l 

The point is on one side of the hyperplane if d is positive, and 
on the other side if d is negative. Which side of the hyperplane 


Vi =Kz, V 2 =Kz 2 V 3 =Kz 3 V N =KZ N 



The Classification Procedure 

In order to be associated with Pattern Class i, a point must 
be on the i side of all boundaries B ij9 j ^ i- If these boundaries 
are implemented, taking proper care of sign conventions, the 
distances from each of them can be combined in an AND 
fashion to arrive at the correct decision. In other words, if a 
positive sign is associated with points on the i side of B ij9 then 
all of the distances in question must be positive if the point is to 
be associated with Pattern Class /. 

Properties of c o 

Bang-bang Case 

Since there are only two admissible values of m(t), there are 
2 N possible control signals. Corresponding to each of these is 
an co sequence, the elements of which define the pattern points. 
For convenience Af = 2 will be used. Any scale factor required 
can later be lumped with the plant. 

As an example, consider the problem for N = 2. There are 
four possible control signals. The four possible control se¬ 
quences, together with their corresponding switching sequences 
and pattern points, are listed in Table 1. 


Table 1 



a 

b 

c 

d 

{m} 

W 

{co} 

hi 
h 0 

hjl, 

i,-i 

hJ2, 

h%[2 hi 

~i> — } 

- *i/2, 

-hJ2 

- hj 2, 

— hJ2 -b hi 


These are shown in Figure 4 along with their associated pattern 
classes. 

It is worth while, at this point, to consider in more detail the 
nature of the decision theory problem being investigated. The 
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unn? U r m , e .f 0ntro1 sisnal at ' = 0+ > W 1> is t0 be decided 
future t M 1 d ,° thlS ’ 0ne must consider the effect of the 

setoasThe S , lgna r’ "* ** -' mN aS weIL 0ne M thus 
c t , Va } ue of mi that value whi ph corresponds to the 

eS f2 sfnce?^ / C ° ntrC>1 SSqUenCe W which minimizes 
'- 2) ' S 6 ‘ hlS decisi0n process is carried out continuously, 

the optimum allowaWe control signal is continuously generated. 

Jfmmiehfh ? SS6S aSS0ciated with ‘he same value 

ot m x might be con S1 dered as a major pattern class, or contro 



Figure 4 . Pattern points and associated pattern classes for bang-bang 
case (N — 2) 



Figure 5. Control boundary for bang-bang case (N = 2) 

class. As an example, the control classes for N = 2 are shown 
in Figure 5 The union of Pattern Classes A and B defines the 

0l aSS ’ SiDC ? ^ P0ints falling « either of these pattern 
classes have m x = + J 2 - associated with the optimum control 
signal Similarly, Pattern Classes C and D define the minus con- 

to w ;= S -r e Z P ° mtS fallklS “ theSC regions correspond 

will T i eP f eCeW i S f liDear b0undary divid “«^se control classes 
will be referred to as the control boundary. Points lying on the 

plus side of the control boundary correspond to m x =+} while 


nts lying on the minus side correspond to m x ~ — \ Since 
the control boundary is determined by the pattern classes, the 
linear decision function defining the various pattern classes 
forms the heart of the control computer. 

Saturation—Limited Case 

, tFls case > ad missible values of m lie in the paralleletope 
denned. by the pattern points associated with the bang-bang 
case*. The interior of this paralleletope will be referred to as 
the null region, R 0 , since points falling in this region result in 

N 

k=i 

For a given z, the problem then becomes that of finding the 
value of m x corresponding to the point in the null region Which 
is closest to z (since minimizing I with respect to o> is equivalent 
to choosing that w which is closest to z). This can best be ex¬ 
plained by referring to the case for N = 2 (see Figure 6). The 
optunum control signal, m la , corresponding to the point z a is 
obtained from the oj x coordinate of the point in R 0 which is 
nearest to z„. The problem has thus been reduced to a geometry 



rnn b f k T R T faCll * ethe SOlution of this Problem consider the 
Control Boundary B in Figure 6. It consists of three segments, 

and . m i IS Proportional to the distance from the ap¬ 
propriate line segment. The proportionality factor is not the 
same in each case. Decision Boundaries T x and T 2 can be used 

19 ^2 z is below T 2 , and B 0 otherwise). 

This represents an extension of what would normally be 
considered to be linear decision theory. Interest lies not only in 
which side of a control bounary z is situated, but how far it is 
from the boundary as well. 

Implementation of z h 

Since zjc — r, c — c M , its implementation can be considered 
m two parts. 

n T1 ,' ie ? r ,°° f ? f this follows from a consideration of (1) and 151 

ss “Kir" •»"* ■>' w„'i s S- 
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Implementation of r k 

In practice, since r lc represents the predicted value of r(t), t k 
seconds from the present, a predictor must be constructed to 
yield this. This paper does not deal with this subject. 


Implementation of c kQ 

Since c k0 represents the solution of the homogeneous differ¬ 
ential equation (HDE) describing the plant evaluated at time t k9 
the implementation is straightforward. For an nth order plant, 
Cj c0 is a linear combination of the output and n— 1 derivatives 
(i.e., the state of the plant). 


Example 1 


G(s) = 


K 

s + a 


HDE c + ac -0 


c 0 (t) — bz~ at — c 0 eT at 

therefore 

Cko — c o e ° tk 


Example 2 


G(s) = 


K 

(s + m 1 )0 + m 2 )’ 


m 1 #m 2 and real 


HDE c + (m 1 + m 2 )c-fm 1 m 2 c = 0 


therefore 


c 0 (t) = K l ^ mit + K 2 t 


-m 2 t 


where 

Co~Ki + K 2 

c' 0 = -m l K 1 -m 2 K 2 

(A ?rt 2 C 0+ c 0 Q~ m i t | + e ~W2f 

™ m, — m 2 


Cl- a — 


m 2~ m l fn E 

m 2 c 0 + c’ 0 tk mtCp + Cp p - mitk 


m? Q~ mtk 


<_J-e 

m 1 — m 2 




m 1 e 


— mitk 


— m 1 — m 2 J 

-K z c 0 + K 4 Cq 


+ £’o 




* - m 2 tk 


-+- 


m 2 — m 1 m 1 — m 2 J 


To be more specific, the case is considered for 

N = 2, t 1 = T = 0-692, t 2 = 2T, a = l 

r(t) will be a Gaussian random process obtained by passing 
white gaussian noise of zero mean and unit power spectral 
density through a simple low-pass RC filter of time constant l/b. 

The four possible control signals, with their associated 
switching sequences and pattern points are listed in Table 1 , 
where h x = 0*5 K 0 and h 2 = 0*75 i£ 0 . 

The pattern points are shown plotted to scale in Figure 7. 
The control boundary is also shown. The plus control class can 
be implemented as the union of the regions to the right of 
boundary B x or J5 0 n J5 2 . 



Figure 7. Pattern points and control boundary for bang-bang example 


Computation of Distance from Segment of Control Boundary 

di represents the distance of a point z from boundary 2J*. The 
sign convention employed results in di > 0 for points on the 
plus side of B^ oc i5 represents the direction cosine in jfth direction 
of boundary B { . 

( a ) Computation for Boundary B x 

D 1 = \bc\= \h\ + (h 2 -h = 0-56 K 0 


Example of Bang-bang Predictor Control System Design 


To clarify the application of the concepts and theorems dis¬ 
cussed thus far, a complete design will be carried out for a first 
order plant with transfer function 


G(s) = 


K 0 a 
s T" a 


a 12 =%-A=045 —=2-2 

1 D x «12 

a 10 = -a 1 2y=-0-H- K o 


The step response, hit ), can be evaluated and is given by 


d 1 =a 11 z 1 -j-a 12 z 2 + oc 10 = 0‘9 z x + 045 z 2 — 0-11 K 0 


h(t) = K 0 (l~e- at ) 

therefore 

h k = h(t k ) = K 0 (l-e-° ,k ) 


(b) Computation for Boundary J5 0 

D 0 = I bd\ = [hi+ (h 2 -2 h yf =0-56 K 0 
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(it is a coincidence that D 0 = Q) 


“oi =-~=0-9 


h 2 -2h 1 


-045 


-=- 2-2 


a OO=0 


2l Control 
logic - 
—■I.... (see 
Figures) 


-1/2 S(s '' s<a 


«o = «o l^i + a 02 z 2 + “oo = 0-9 z 2 - 0.45 z 2 
(c) Computation for Boundary B 2 
D 2 = D 1 = 0-56 K 0 

a 2i = au=0-9 —=1.1 


(X 22 ^12 — 0*45 ___ 2,2, 

X 12 

x 2o=-cc 10 =O.U K 0 

^2 =a 2 i^i + K 22 z 2 +a 20 = 0-9 + 045 z 2 + o-l 1JC 0 

The required logic is shown in Figure 8. 

Implementation of z k 

xs e P rxxi sf p- 

*' W -=>TP 


Since 0 r (w) can be realized by exciting an RC filter with 
tae cons'*.' 1 /* by whiK „ oise from , J* o 

Xalh'Th,! f “ r b " hown “ '« » simple attenuator 
Jmysically, this answer can be obtained by noting that since 

£ sxrexxfrx?* s r ■? 

rXLXfr, r f r r »“■ 

ee - The “ functi ° n ° f 

f > k ==e k , therefore r k = r n e ~ iffc 


Figure 9. Complete block diagram for bang-bang example 

exDectatinn S ^jf ussian ’ th ? s Prediction represents the conditional 
expectation of r k given the history of r(t). 

t] ?‘ The signa1 ’ is simply obtained from the solution of 
For Z 0 ” X"“ Ascribes the p, mt . 


J-iiUS _ _ 

^io — 0*5 c 
c 20 = 0*25 c 

fnFtZTef. bI ° Ck f ° r the SyStem ” question is sh own 

SatSon f PrediCt ° r C ° ntro1 Sy5tem Des ign Subject to 

For purposes of comparison, the plant, input and Dara- 
sXXi? T '“<» °P predictor control 

useZo 1X2 X be * he “ ime a! M 

systems> the design of bang-bang predictor control 

G(S)= S N=2,a=l, t t = T=0-692, t 2 =2 T 

The pattern points in this case are the same as in the bane- 
bang case (this will always be true). Figure 10 indicates * and 


R V—VsA, 
B, 2’2 
2 2 o-WV 


M ] d 2 


R *r—' v v\ 
B 2 2-2 



J% “ re Got*®/ logic for bang-bang example 


Figure 10. Null region and control boundary for saturation-limited 

example 
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Figure 11. Block diagram for example of saturation-limited design 


the new control boundary. The oc t for the various control 
boundary segments have already been computed so the results 
will merely be presented here. 

do = z i 

di = 0*9 + 0*45 z 2 --0*11 Kq 

d 2 = 0*9 Zi + 0*45 z 2 + 0*11 K 0 

In these expressions, the sign convention is such as to make 
dt > 0 for points on the plus side of the control boundary. 

Decision Boundaries T x and T 2 are specified by the pattern 
points through which they pass. The resulting expressions for 
dp\ and dp 2 are 

d Tl = 0*45 z x - 0*9 z 2 + 0*225 K 0 
d T2 = —0*45 z x +0*9 z 2 + 0*225 K 0 


Case 2 represents an unstable second order plant. The unit 
step response (for N = 2) is shown in Figure 13. This run was 



In these expressions, the sign convention is such as to make 
dTi > 0 for points on the i? 0 side of 7$. 

The implementation of z jc is exactly the same as for the 
bang-bang example. 

Figure 11 is a block diagram of the complete system. 

Results of Computer Simulation 

An IBM 650 digital computer was employed to evaluate the 
step response of a predictor control system with a saturation- 
limited second order plant. Two specific cases were considered. 


1000 


G ' (s)= ?dSw |m<,)|£ T ;{ - n2J 


Figure 12. Step response of predictor control system with lightly-damped 
saturation-limited second order plant 


c (t) 



Case 1 represents a lightly damped but stable second-order 
plant. The unit step response (for N = 2) is shown in Figure 12. 


Figure 13. Step response of predictor control system with unstable 
saturation-limited second order plant 
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repeated with noise added to the plant input and the plant out¬ 
put remained essentially unchanged. 

The work in this paper was performed at the Polytechnic 
Institute of Brooklyn as part of a Doctoral Dissertation and was 
supported in part by the Air Force Office of Scientific Research 
under grant number AFOSR-62-280. 
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DISCUSSION 


P. Dorato, Polytechnic Institute of Brooklyn, Brooklyn, U.S.A . 

In order to understand some of the limitations of the method of de¬ 
sign presented by the author, it is useful to outline the difference be¬ 
tween the problem which must actually be solved and the problem 
which the author refers to as ‘the hypothetical system’. The actual 
problem contains the following elements: 

(1) A linear plant with a continuous-time input m (t) and a continu¬ 
ous-time output c (0 with dynamics 

c h (t — r) m (t) dr 

(2) A performance index 

(Odf|pJ 

Where e(t)^c (t) — r (t), r ( t ) is, in general, some random process, 
and P represents the past data on which the conditional expectation 
E is conditioned. 

(3) Control input constraints 

\m{t)\<L 

The hypothetical problem contains the following elements: 

(а) A linear plant as above, with the only difference that now m (?) 
is constrained to be piece-wise constant, with changes in m (?) occur¬ 
ring only in intervals of T sec. 

(б) A performance index 


1 1 

m (0) = y Sgn ip (0), for | m (01 = y 

m(0) = -L S at <p(0),for|m(?)|<T 

^ 2 

The author s realization of the optimal controller for / = 0 is quite 
simple (for the hypothetical system the control law is open-looped 
since the values of m (0) and m ( t ) depend only on data obtained for 

The author then uses this control law obtained at time t = 0 for 
the control of the actual problem for all time. The simplicity of the 
solution and the degree of similarity between the actual and hypo¬ 
thetical system make this design technique quite attractive. However 
the following problems do arise in the application of this technique 
to any given problem. 

(la) The final control system may not even be stable, much less 
optimal. 

(2d) If the final system is stable, what is an optimum choice for 
the design variable T? 

Because of the complicated non-linear nature of the final control 
system it is difficult to study the above problems analytically. Thus in 
the application of the above design procedure to any particular system, 
some experimental studies should be made on the problem of stability 
and on the ‘optimal’ choice of T. 


S. Horing, in reply 



/=£ {[z (T) — co (T)] 2 + [ z (2 T)- co (2 T)f \P} 
where z (t) — c 0 (t) — r (t) and where 


co(t) = 


h(t — x)m(T) dr 


(c) Control input constraints 


1 


l m (OI=y and |m(t)[<l 

The author then makes use of some basic results in linear decision 
theory to solve the hypothetical problem. However, only the optimal 
control input at time t = 0 is obtained, i e 


In reply to the comments made by Dr. Dorato, it should be noted that 
a detailed investigation of the stability of the resulting system has not 
yet been made. Neither has an optimum choice for the prediction time, 
T, been found. For the saturation-limited case, however, approximate 
procedures for overcoming these limitations do exist. Both procedures 
are based on quasi-linearization techniques which restrict consideration 
to points falling in the null region. Under this condition, the system 
behaves m a linear manner and linear stability techniques can be suc¬ 
cessfully applied. This is a ‘small signal’ approach, but appears to be 
reasonable. The selection of the prediction time, T, can also be made 
under the same conditions. If this is done, the procedure would merely 
require the specification of a performance index for the overall system 
and the selection of T so as to optimize the system with respect to 
this index. 
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Summary 

A method of optimal control for a class of any-order non-linear 
systems is described. The method is based on prediction of the optimal 
motion of all the system coordinates. Prediction is realized by for¬ 
mulating the optimal motion of a steadily decreasing number of 
elements of the plant at steadily increasing speed, using high-speed 
iterative computers. Some principles of the design of the predictors 
are presented, and experimental results for a fourth-order plant are 
given. 

Sommaire 

Une methode d’optimalisation d’une classe de systemes non-lineaires 
d’un ordre quelconque, est decrite. Cette methode est basee sur la 
prediction d’un deplacement optimal de toutes les coordonnees du 
systeme. Cette prediction est realisee en formulant la condition 
d’optimalite d’un nombre decroissant d’elements du systeme a une 
vitesse croissante et en utilisant pour cela des systemes numeriques 
iteratifs a tres grande vitesse. Les principes de realisation de 
tels systemes da prediction soul decrits et des resultats experimenlaux 
de leur application a un systeme du 4e ordre sont donnes. 

Zusammenfassung 

Eine Methode fur die zeitoptimale Regelung einer Klasse von mcht- 
linearen Systemen beliebiger Ordnung wird beschrieben. Die Methode 
beruht auf der Vorhersage der gunstigsten Bewegung aller System- 
koordinaten. Die Vorhersage erfolgt durch Formulierung der giinstig- 
sten Bewegung fur eine standig abnehmende Anzahl von Gliedern der 
Strecke bei standig zunehmender Geschwindigkeit; hierzu werden 
schnelle iterative Rechner verwendet. Die Arbeit enthalt einige Aus- 
fuhrungsprinzipien fur die Vorhersageeinrichtungen sowie Versuchs- 
ergebnisse fiir eine Strecke vierter Ordnung. 


Introduction 

In view of the increasing demands which arc made on the 
quality of automatic control processes, more and more use is 
being made of optimal control systems, particularly of a wide 
class of time-optimal systems. 

The development of time-optimal systems is at present badly 
hampered by the difficulty involved in designing the controlling 
part of the system, which, except for the simplest cases of 
second-order linear plants, involves the use of multivariable func¬ 
tional generators, or of complex boundary-problem computers . 

Because of this, there has recently been a search for new 
approaches to the design of optimal control systems. In this 
connection mention should be made of the work of Coales and 
Noton 2 , who proposed that search of the switching moment of 
the control action should be realized on the basis of high-speed 
examination of a family of phase trajectories (future behaviours 
of the plant), on the assumption that this switching will take 
place at some future moment. Chestnut, Sollecito and Tioutman 

* General principles of this method were established by the 
authors in cooperation with Professor A. Ya. Lerner. 


further developed this principle 3 , substituting for search of the 
switching moment a step-by-step analysis of a family of the 
phase trajectories, also obtained at high speed, on the assumption 
that switching of the control action has taken place at the 
current moment of time; the actual switching is executed when 
the predicted trajectory passes through the origin of the coordi¬ 
nates. Characteristic features of the above works are: (a) Predic¬ 
tion by repetitive computers of the set of future optimal 
behaviours (trajectories) of the plant, with verification of each 
trajectory to see whether it corresponds to the assigned boundary 
conditions; (b) the use in the control system of a logic for 
second-order plants, provided for not more than one switching 
of the control action. The latter confines the possible applica¬ 
tions of these methods to second-order plants, or to plants 
reducible to the second order, having no supplementary con¬ 
straints on the coordinates. 

It is, however, possible to remove these constraints, at least 
for plants consisting of a number of series connected first-order 
elements, linear or with mono tonic non-linearities (or plants 
reducible to such a form), by using the peculiarities of the struc¬ 
ture of optimal processes in such systems. In this way it is 
possible to realize an optimal control system for a plant of the 
nth order, having an optimal controller for a plant of the (n-l)th 
order and a predictor. Applying the same principle in succession 
to plants of the (n-l)th, («-2)th, ... orders, up to and including 
the second order, it is possible to construct an optimal control 
system for an nth order plant, the controlling part of which 
will consist of a set of predictors. 

Construction of Optimal Control Systems by Successive 
Reduction of the Order and Prediction 

Considered here are plants described by a system of differen¬ 
tial equations of the form: 

Xi —f i , u) 

l) W 


X n f n (A]3 _ l) 

where u = u (t) is the control action, while 

K0I<1 

All the functions ft are assumed to be continuous and continu¬ 
ously differentiable with respect to and x<n, while A is 
continuously differentiable with respect to u, and the partial 
derivatives bfjbx^ and bfjbu do not change sign throughout 
the domain of variation of the variables in question. 

Moreover, on some x k , there can be imposed constraints 

of the form: ^ 

\x k \<x k 
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specifying the permissible domain of states of the system in the 
phase space. The problem is to synthetize a control system 

mS * e time -°P timal shift of plant (1) from any 

nitial state to any assigned equilibrium state. 

o solve this problem, use will be made of a property of the 
structure of optimal processes in plants of type (1), namely that 
he trajectory of the optimal process consists of successive 
sec ions, on each of which the control corresponding to it 
coincides with the optimal control for a type (1) system having 
an order lower by a unity than the initial one. For example if 

rwitWw 11 iS possible t0 ensure, for a (»-l)th order plant 

[without the last element in (1)], a control action which in 
minimal time imparts to the coordinate x n - t , an extremal value 
. ing into account the imposed constraints), and which then 
in minimal time, transfers the plant to the assigned state (with 
respect to the « - 1 coordinate), and if, moreover, the coordinate 
a ' ,! y eaches ‘he assigned value at the final moment, then the 
control action and corresponding trajectory of the whole 

^ S r, m 4 ( i ) T time -°Pt™al. Th e proof of this is given by 
k I ko (for the case when constraints of the T k type are lacking), 
where it is shown that such a control action satisfies Pontryagin’s 
Maximum Principle 5 . 6 


Figure 1 is the block diagram of an optimal control system 
based on these principles. The scheme consists of three main 
parts: the plant itself with an optimal controller [for the (n-l)th 
order], the predictor (P), and the logical gate (L). 

The optimal controller, in the plant assures optimal motion 
ot the (n-l)th order plant towards the value of the coordinate 
Xn ~ 1 assigned by the logical gate L. The predictor is a high 
speed repetitive computer, which simulates the plant together 
With controller optimal for the n — 1 coordinate, the setting 
of which agrees with the assigned value of the coordinate x„~,. 



Obtaining, at the beginning of every cycle of the solution, data 
on the state of the plant (the values of its current coordinates), 
the predictor computes the value which will be reached by the 
coordinate x n if, starting from a given moment, the truncated 
(n-1) order system is brought to assigned equilibrium state in 
minimal time. 

The output signal of the logical gate ul is determined by 
the difference between the assigned value of the nth coordinate 
x n spec and its predicted value x npred , from the equation: 

- x *_! sign (x„ spec - x„ pred ) • sign Ml. 

ox n „ x 

U L whe n x„ pred /x„ spcc ( 2 ) 

~ 1 s r ec when x„ pred = x n spec 

When x mspec is applied, a difference arises between x n spec and 
x?ipredj as a result of which the logical gate L, in accordance 
with (2), sends to the optimal controller a signal for the varia- 
tion of the coordinate x n - lt (taking into account the sign 
of the difference). Moreover, the predictor continuously 
calculates the value which will be taken by the coordinate 
*» lf > at the gwen instant, the setting of the optimal controller 
is switched from x rl to spec . As soon as the value of 
X n pred reaches x n spec , the logical gate will bring about an actual 
change of the setting of the controller, after which the system 
will adopt the specified position, under the influence of the 
optimal controller. The predicted value x n pred remains unaltered 
over this interval of time. 

The chain of reasoning employed for the synthesis of an 
optimal system of the «th order can be used to synthetize an 
(ft-l)th order optimal controller for plant and predictor; that 
is, recourse is made to a system with an (n- 2)th order optimal 
regulator and two predictors for the coordinates x n and x n ~ x . 
Naturally, in this case, the predictor which works out the future 
value of the coordinate x n , and which itself forms part of the 
predictor for the coordinate x n9 must operate at a higher speed 
than the latter. Applying this method successively a further 
« - 3 times, one arrives at an optimal control system con¬ 
taining n - 1 predictors (P 1; P 2 ,.. p n _ x ) witll their correspond¬ 
ing logical gates (Pi, L 2 ,..., L n -j), but containing no other 
optimal controllers (Figure 2). It is a characteristic feature of 
this system that the optimal nature of the calculated trajectories 
m any of the predictors is ensured by the presence in the make-up 
of any of them of other predictors which calculate the motion 
of a successively abbreviated number of elements at ever- 
increasing speed. Figure 3 is a block diagram illustrating the 
method of synthesis of the predictors. 

To solve a tracking problem by the method described, 
recourse must be made to error equations, as was done by 
Coales and Noton 2 , and by Chestnut et al. z . 


Optimal Control of a Fourth-order Plant 

To illustrate the method, Figure 4 shows the example of a 
time-optimal control system for a fourth-order plant consisting 
of four integrating elements. The system contains three predic¬ 
tors: P l9 P 2 and P 3 . Let there be supplied to the system at some 


Figure 1. Block diagram of an automatic control system with prediction 
for one coordinate 


* If no constraint x n - 1 is given, then the greatest value of the 
coordinate x n _^ that can be physically represented is introduced into 
the logic block in its place. 
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„ tbP, . Pn.^oP, . P n _, tO P k . 2 ...P nJ t0P h _,. P nJ t0P n 2 . P n _, to P n _, 
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xiu wVixk 


i u L(W) 

L_ruTT*-~ spec 
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x n-1 pred 
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Figure 2. Block diagram of optimal control with prediction for n — 1 coordinate (the general case) 
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Figure 4. Automatic control system for fourth-degree plant —^ 
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moment a ‘specification’ concerning the coordinate x 4 . If at 
this moment the state of the system is such that, with optimal 
alteration of the coordinate x 3 to the specified value correspond¬ 
ing to an equilibrium state (i.e., to zero), the coordinate x 4 does 
not reach the specified magnitude, then as a result of the 
difference between x 4spec and x 4pred the logical gate L, gives a 
signal ul 3 , corresponding to the limit permissible value of the 
coordinate x 3 , with the appropriate sign determined by the 
direction of the ‘acceleration’ of the coordinate x 4 . If there is 
a difference between u Ls and x 3pred , logical gate L 2 gives a 
specification U L2 for the variation of the coordinate x 2 , and 
logical gate £j, under the same conditions, gives a specification 
u Ll for a variation of the coordinate x 4 which switches the 
control relay. After this the system begins to ‘accelerate’ with 
maximum speed in the required direction, and at some moment 
the value of x 4pred becomes equal to the given value x 4spec . 
Starting from this moment, the coordinate x 3 must be zeroed 
with optimal rapidity, so logical gate L 3 changes the command 
signal «£ S to zero. In the same way, logical gates L 2 and L, 
change the signs of u L2 and u Ll , and the first switching of the 
command relay takes place, after which the system starts to 
brake with respect to the coordinate x 4 . During the rest of the 
process, x 4 pred will equal x 4 spec . When the value of x 3 pred 
reaches zero, gate L 2 issues a command for optimal change of 
coordinate x 2 to zero. This command enters the relay via gate L, 
and effects the second switching. After this, x 3 pred remains equal 
to zero. The third switching occurs in the same way, when 
*2 pred = o. This last interval ends when x 4 reaches equilibrium 
value (zero). Thus by the end of the process x 3 = x 2 = x 4 = 0 
and x 4 = x 4 spec . If, in the course of the process, the predicted 
value of some one of the coordinates, for example x 3 , reaches a 
magnitude equal to the limit value prescribed for it by logical 
gate L 3 , the specification for x 2 will be switched to zero by the 
gate L 2 , and the system will start ‘braking’ with respect to the 
coordinate x 3 . At the end of this ‘braking’ process, x 3 reaches 
its permissible value, and maintains it until the predicted value 
of x 4 reaches the specified value. In this case the process will 
consist of a greater number of switchings, as follows from the 
theory of optimal control 8 . The results of simulating control 
processes by the proposed method are shown in the oscillograms 
of Figure 5 (a), ( b ), and (c), which show processes for the cases 
of no constraints on the phase coordinates, and the application 
of constraints on the third coordinate and the second and third 
coordinates together. Oscillograms of the outputs of the 
predictors P 3 (x 4 pred ) and P> (x 3 pred ) are also given. 

If it is possible to realize optimal control of the first k 
elements of the plant {k = 2,3,.. .) in some other way, the 
number of predictors can be reduced by k — 1. The first of the 
remaining predictors must include a model of the corresponding 
optimal controller, the second a model of the first predictor 
and so on, as has been described for the general case. 

Some Features of the Design of Repetitive Analogue Predictors 

- ^ P redictor is a high-speed iterative computer, operating 
with acceleration of the processes (with respect to the plant). 
Because of the need for a high repetition rate, while the require- r- 
ments for accuracy are relatively low, the use of analogue 
principles m the design of the predictors is most expedient. f 

The repetition rate is selected from considerations of the „„ 

increment of the predicted quantity permissible, for reasons of 


accuracy, in a cycle of the solution. The time scale is chosen 
with references to the repetition rate and the duration of the 
processes in the plant to be predicted by the device. 

A predictor usually consists of the following main units: 
an analogue of the relay device supplying the control action- 
repetitive analogue computing elements (linear or non-linear),’ 
with a wide pass-band; a memory element, which stores informa¬ 
tion between cycles of the solution, and, finally, a control system, 
which provides the necessary sequence of switching operations’ 

The analogue relay element is usually a flip-flop or opera¬ 
tional amplifier with a limiter in the feedback circuit or on 
the output. 



o u'z 0-4 0-6 0-8 1-0 * x 



0 0-2 04 0-6 0-8 1-0 (b) 



0 0-2 0-4 0-6 0-8 1-0 “ ^ 


Figure 5. Oscillograms of transient processes in the optimal control 
system of a fourth-degree plant: 

(a) with no constraint on the phase coordinates; (b) with a constraint 
on the coordinate x 3 ; and (c) with a constraint on the coordinates 

x 2 and x 3 
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A METHOD OF OPTIMAL CONTROL PREDICTION 


The amplifiers for the computing elements must ensure the 
desired accuracy of operation, and must have low drift. For this 
purpose, according to Polonnikov 7 , the most suitable is a direct 
current amplifier with a zero drift compensation network based 
on the ideas of Prinz. Figure 6 gives the structural scheme of a 
scale computing element and integrator. Here in the RESET 
cycle all the switches are closed, and because of the full negative 
feedback the capacitor C k is charged up to the drift voltage at 
the amplifier output. In the ‘solution’ cycle the switches are 
opened, and the compensating voltage across the capacitor C k 
is connected in series with the voltage at the summing point. 
Ordinary switches of the bridge type are used. 



(a) 



Figure 6. Zero drift compensation circuit of computing elements, 
(a) Scale computing element , and (b) integrator 

The memory element stores the solution at the end of an 
operational cycle for the duration of the RESET period of the 
next solution cycle. It can be constructed from a combination 
of operational amplifier, memory capacitors and switches, for 
example. 

The control system produces clock pulses which control the 
mode of operation of the switch, and can be constructed using 
conventional multivibrators. 


Conclusions 

The theoretical possibility of constructing time-optimal 
control systems for wth order plants has been demonstrated, 
using a set of predictors as the optimal controllers. 

Such optimal control systems are synthetized according to 
a proposed way, on the basis of a mathematical description of the 
plant. Elaborate calculations are not required, and the adjust¬ 
ment of the control system is simplified. 

The predictors used in the control system can also function 
as ‘advisors’ to the operators in the case of manual control. 
For third-order and fourth-order plants, the method described 
can be realized with the aid of a comparatively simple apparatus, 
which can be built with the technical means now available. 

The extension of the method to other, more complex, 
optimal control problems will require further investigation of 
the structural features of optimal processes, and the development 
of very high-speed and reliable means of mathematical 
simulation. 
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DISCUSSION 


I. F. Coales, The University , Cambridge 

This is an interesting paper giving a rigorous method for designing a 
time-optimal control of an «th order system using predictive control, 
but, intuitively, with random inputs. I cannot see any need to go 
beyond prediction of the first derivative of error as was proposed 
in our original paper 1 . 

Experiments since carried out in various places have shown that m 
a number of practical cases the improvement resulting from predicting 
the second derivative as well as the first has resulted in only an in¬ 
significant improvement in performance. 

There is plenty of practical evidence that predictive control can be 
used successfully for higher-order systems, both linear and non-linear, 
provided that the fast model of the plant is a sufficiently close approx¬ 
imation to the actual plant. Of course, when one comes to higher-order 
systems, care has to be taken to ensure that the iterative process is 
convergent, since otherwise the system may become unstable. 

The object of work on predictive control is, in my view, to find 
iterative methods of control for complex systems with randomly chang¬ 


ing inputs and parameters which will give sub-optimal solutions of 
adequate performance that are relatively easily realized. 

Reference 

1 Coales, J. F. and Noton, A. R. M. An on-off servomechanism 
with predicted changeover. Proc. Inst, elect. Engrs Pt. D, 103, 
No. 10 (1956) 449-462 

B. Ya. Kogan, in reply 

Optimal-response control of high-order systems based on the logic of 
optimal control of second-order systems can be effected only for those 
particular cases when there are two dominant time constants in the 
system, and the remaining time constants are so small that they can 
be considered parasitic, i.e. for systems practically reducible to second- 
order systems. 

In our paper we pose the problem of how to achieve optimal- 
response control with the aid of predictors for high-order systems not 
reducible to second-order systems. 
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In contrast to the method of Coales and Noton 1 , the method ex¬ 
plained in the paper does not require an extra adaptive system, the 
stability of operation of which depends on the order of the equation 
of the plant, and therefore with a rise in the order of the system, 
difficulties do not arise with the stability of the loop producing the 
switching moment. 

Here difficulties arise in connection with the increase in the volume 
of equipment and the requirements for its speed of response. 

It has recently been possible to prove that the method explained in 
the paper can also be extended to plants containing oscillatory 
elements 2 . 

I must agree that in further studies of control using predictors, 
attention must be paid to complex systems controlled by randomly 


varying input signals against a background of interference, and also 
to the creation of close to optimal control systems with the aid of 
predictors. 

This will make it possible to simplify control equipment consider¬ 
ably without noticeable impairment of the quality of operation. 
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Optimalization of Non-linear Random Control Processes 

R. KULIKOWSKI* 


Summary 

There are known controlled systems such as chemical plants or air¬ 
planes whose differential equations are not known completely to the 
controller because of environmental changes, ageing, etc. In many 
systems of this kind the best which can be accomplished is to con¬ 
struct a multistage optimalizing process which converges to the opti¬ 
mum control. In the case of zero-memory, non-linear plant processes 
of this kind can be constructed if the gradient of the performance 
measure is known or can be determined experimentally. 

In this paper an extension of this method is considered for the case of 
non-linear plants having memory and changing randomly in time. In 
the two introductory parts of the paper the assumptions and the neces¬ 
sary and sufficient conditions of optimality are formulated. It is shown 
that at any stage of optimalization the best change of the input signal 
should be adjoint to the mean value of the gradient of the performance 
measure. Then an optimalizing process, based on the so-called fixed 
point theorem, is constructed. It is shown that for certain classes of 
non-linear random plants all the necessary information about the 
generalized gradient can be obtained experimentally. As an example 
an optimalizing process which minimizes the cost of input energy and 
maximizes the output gain of a non-linear plant has been constructed 
and is discussed. 

Sommaire 

On sait qu’il existe des systemes k optimaliser, tels des usines chimiques 
ou des avions, dont les Equations differentielles caracterisant leur 
comportement ne peuvent etre completement predeterminees car 
elles dependent de circonstances exterieures au systeme et varient 
continueliement. La meilleure fa$on de proc6der est alors d utiliser un 
procede d’optimalisation par approximations successives qui con¬ 
vergent vers un regime optimal. Des dispositifs d’optimalisation 
depourvus de mdmoire peuvent etre alors realises si le gradient 
caracterisant la performance k atteindre est connu ou peu etre 
determine experimentalement. Ce rapport considere une extension de 
cette methode pour le cas de systemes non-lineaires avec memoire, 
et evoluant continueliement de fapon aleatoire. 

Les 2 premieres parties de ce rapport indiquent les hypotheses et 
les conditions necessaires et suffisantes d’une telle optimalisation. 11 
est montre qu’a chaque dtape de cette optimalisation, la meilleure 
variation du signal d’entrde doit etre ajoutee a la valeur moyenne du 
gradient de la mesure de la performance. Un processus d optimalisa¬ 
tion base sur le theoreme du ‘point fixe’, peut etre alors realist. Le 
rapport montre que pour certaines installations non-lineaires de 
caractere aleatoire, les informations necessaires pour la determination 
du gradient generalist, caracttristique des performances a atteindre, 
peut etre obtenu exptrimentalement. II dtcrit et discute un exemple 
de processus d’optimalisation destint k minimaliser le cout de 
Ptnergie d’entree, et k maximaliser le gain de sortre d’un systeme 
non-lintaire. 


Zusammenfassung 

Es gibt Regelstrecken, so z. B. chemische Anlagen oder Flugzeuge, 
deren Differentialgleichungen fur die Regelung nicht vollstandig 
bekannt sind, da Anderungen der Umgebungsbedingungen, Alterung 

* This research was partially supported by the National Science 
Foundation under Grant NSFG-14514. 


usw. vorliegen. Fur viele solcher Systeme ist der Aufbau eines mehr- 
stufigen Optimierprozesses, der nach einer optimalen Regelung strebt, 
das Beste, was sich erreichen laBt. Im Falle nicht-linearer Regel¬ 
strecken ohne Gedachtnis konnen Prozesse dieser Art aufgebaut 
werden, wenn der Gradient des Gutemafies bekannt oder experimented 
bestimmbar ist. In diesem Beitrag wird eine Erweiterung. dieser 
Methode betrachtet, und zwar fur den Fall, daB die' mchtlmearen 
Regelstrecken auch Glieder mit Gedachtnis enthalten und sich ini 
Laufe der Zeit zufallsbedingt andern. In den zwei einfuhrenden Teilen 
des Beitrages werden die Voraussetzungen sowie die notwendigen und 
hinreichenden Bedingungen fur das Optimum formuhert. Es wird 
gezeigt, daB bei jeder Stufe der Optimierung die beste Anderung des 
Eingangssignales adjungiert zum Mittelwert des Gradienten des 

GiitemaBes sein sollte. „ , „ , 

Ein Verfahren zur Optimierung wird auf der Grundlage des so- 
genannten „Festpunkttheoremes“ entwickelt. Es zeigt sich, daB fur 
bestimmte Klassen nicht-linearer zufallsgestorter Regelstrecken alle 
notwendigen Informationen liber den verallgemeinerten Gradienten 
experimentell beslimmt werden konnen. Als Beispiel wird em Opti- 
mierverfahren abgeleitet und diskutiert, der auf ein Minimum der 
Kosten der Eingangsenergeie und auf ein Ertragsmaximum bei einer 
nir'iifJiniafirf'n Sltrecke fiihrt. 


Introduction 

In the theory of optimum control systems it is usually as¬ 
sumed that the plant differential or operator equations are 
completely known to the controller. In such cases, through the 
application of known optimalization techniques, the optimum 
control signal can be derived by an analogue or digital computer 
and applied to the plant during any time interval. However, 
there are known systems such as chemical plants and aircraft 
whose differential equations are not known completely to the 
controller because of environmental changes, ageing, etc. In 
many systems of this kind the best that can be accomplished is to 
construct a multistage optimalizing process which converges to 
the optimum control. All the necessary information for the 
construction of such a process can be obtained by observing 
outputs of the plant at every stage for known inputs. Applying 
this approach to non-linear, zero-memory plants, the gradient 
of the performance measure can be determined and known 
iteration methods, based on the gradient concept (such as 
steepest descent, non-linear programming, contracting iterations, 
Newton method, etc.) can be applied 1 . 

If it is desired to extend these methods for the case of non¬ 
linear and random plants having memory (i.e., possessing 
inertial elements), with the object of obtaining a stable opti¬ 
malizing process, one should first define and determine ex¬ 
perimentally the generalized gradient of the performance measure 
and then construct the convergent iteration process. It will be 
shown that these problems can be solved successfully, at least in 
the case of certain classes of non-linear, inertial, random plants, 
by using some concepts of non-linear and probabilistic functional 
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analysis. However, since the author realizes that one of the main 
purposes of a short technical paper is to present the arguments 
and results in a form which is understandable for the majority 
of engineers, an attempt has been made to avoid abstract 
formulations. The more delicate, formal questions are therefore 
explained in Remarks I, //, III which can be omitted during the 
first reading. 


Assumptions 


(1) It will be assumed that the controller generates signals 
x(t) which may be subject to certain constraints, such as 
volume or energy constraints, i. e. 


\x{t)\ p dt<L= const, where p = l,2 


( 1 ) 


or amplitude constraints, i.e. 


max \x(t)\<M= const, etc. (2) 

The controller can observe the output y (?) of the plant for 
every x(?) applied to the input by the feedback loop (see 
Figure 7). 


where G [x, y, y d ] is a known, twice differentiable function of the 
arguments x, y. 

As an example, consider a chemical plant (for instance, a 
reactor, distillation column, etc.) described by the positive 
operator A (x) [which is non-negative for any x (?)]. The amount 
of steam, fuel or electrical energy delivered to the plant within the 
time interval [0, T] will be equal to J J | x (?) | v d t, where p =-- 1 
or 2. The output product obtained in time T from the plant will 
be Jo ^ (x) dr. Then as the cost of running the plant in the time 
T, we can take the performance expression 


F(x)=l 1 


:(t)| p dt 


a: 


A (x) dt 


(7) 


where 1 2 — positive coefficients which express the cost in the 
accepted currency. 

As the next example consideration can be given to an 
autopilot-controller, which minimizes the integral error between 
the desired y d (t) and the actual y = A (x) path angle of an 
aircraft: 


r +t 

F(*W o w ( t )\yAt)-A(x)\ p dt (8) 





k y (t) 


Figure 1 . The optimalizing control system 

(2) We shall also assume that the form of the output-input 
relation of the controlled system can be described with sufficient 
accuracy by a non-linear, twice differentiable, integral operator 
This operator for example, may be of the polynomial type: 

y=A(x) = A 0 (t) 


where w (?) — the given weighting function and x (?) is subject 
to the amplitude or integral constraints (2) or (1). 

In the case where we want to minimize the final deflection 
£ (n = Vd (T) — y (T) and its derivatives £<*) (?)| f=r , i.e., when 


F(x) = A £ A t 
1 = 0 

the weighting function 


dt 




t=T 


(9) 


w(t)= £ AiC-iys^cr-t) and p =1 


+ z 
1 = 1 


k i(t ; Ti... T i)X (Tj)... x (-?,.) dr* ... dt; (3) 

where the kernels k { and the function A 0 (?) are generally un- 
known to the controller. 

The differential dA (x, h) of the operator A (x), which is an 
extension of the usual concept of the differential of a function 
can be defined as 5 


dA (*» h )=lim — {A [x (?) + yh (?)] — A [x (?)]} 

=^[*(0+yM0],- o 


(4) 

IsTdl ( w S ^ arbitra 7 function sub J ect to the same constraints 
as x(?) We assume also that it is possible to determine the 
approximate value of (4) experimentally by obserXg the 
outputs of the plant for x (?) and x (r) + y h (t J and 

7 ' A [ x (0 + 7h(0]~A [x (0]} *<L4 (x, h) (5) 

where y is a sufficiently small number. 

(3) It is assumed that a performance measure F(x) is given 


rr 

^)=Jo G [x,y,yi\dt 


( 6 ) 


should be substituted into (8). The problem becomes more 
complicated when one wants to minimize the time T subject to 
the constraints eW (T) = 0, and (1) or (2). 

(4) In the general case A (x) may be a random operator, i.e., 
for the same x (?) it may be the case that y (?) == A (x) is a 

f ™ CtJOn ' Therefore ’ in the Performance measures (7), 
(°)> (°) the expected values will be assumed i e E i A fAl 
instead of A (x). ’ ' ' 1 >s 

In many cases y d (?) is not known a priori and the transient 
erm A 0 (?) caused by the non-zero initial conditions of A (x) is 
not known as well. Therefore in the case of (8), (9) it will be 
assumed that the function y a (t)-A 0 (t) can be predicted so 
hat it will be known, at least approximately, in the interval 

JJjP T^ A 1 X) Wl1 ” 0t depend 0n the initial conditions. In the 
case of (7) the output y (?) is the sum of the processes due to 

™'.'™ [»• U »d «(,) acting in the pastanj ,£ 
last term contributes to the output. 

Now the goal can be formulated, and it is necessary to find a 
« Jgna. *«) which will tnininfe the 

termW tf * ' a ° ^ t0 S ° lve this probIem one has to de- 
termrne the conditions of optimality and construct the opti¬ 
malizing process which will converge to the best x (?). 

Remark /. Speaking more precisely, one wants to minimize 
the twice-weakly differentiable functional F(x), determined on 
the open or closed sphere of/® [0, T] space 

IWh(f o l < p 
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OPTIMALIZATION OF NON-LINEAR RANDOM CONTROL PROCESSES 


The norm ||x|| in the space L® should be defined as the 
so-called ‘essential maximum’ or 

lix||=infs sup |x(f)|>, mes£=0 

E (te[0, n-E ) 

which is, roughly, equivalent to (2). The functional (9) should 
be regarded as the so-called Schwartz distribution or generalized 
function. The <5® (0 functions can then be defined as the limits 
of weakly converging linear functionals. 

The concept of a random operator is based on the notion of 
the so-called generalized random variable 2 . Usually, in control 
theory, random phenomena are described by random numbers 
or stochastic processes, which are, roughly, random numbers for 
any fixed time moments. It is known that the random numbers 
can be defined in the axiomatic way as the mapping of the space 
of events into the space of real numbers. It is possible to extend 
the notion of random numbers to the generalized random 
variable, which is a Borel measurable mapping of the space of 
events into some topological or metric space (in our case only, 
a sphere of L v [0, T] space). More precisely 2 , let (13, S') be a 
measurable space and X a non-empty metric space with the 
o--algebra Z of all Borel subsets of the space X. Then the mapping 
V of the space Q into X is called a generalized random variable 
if the inverse image under the mapping V of each Borel set B 
belongs to the cr-algebra S, or in symbols, if {[co:V (a)) e B ] 
: B e Z} c S. 

The random operator, which can be denoted by A ( co, x), 
co e 13, can be defined as the operator which for every fixed x 
is a generalized random variable. 

The expected value of A (co, x) can be defined as the Bochner 

integral over the space Q: 

£{M(x)} =a| A (oi, x) d,it (co) 


assuming that the second differential d 2 /dy 2 F [x + yh ]| y=0 1S 
positive for all h (t). 

It is more convenient to formulate this condition in a form 
which does not depend upon the arbitrary function h ( t ). If an 
example is taken of the operator 

J 4 ( x ) = | k t {t-x) | dx^ dr (11) 

which has the following differential 


Eqn (12) 

one can substitute (12) into (10) and by interchanging the 
integration order there is obtained 


d F (x,h) — 


where 


h ( t ) d t {2 lx ( t ) - cL4* (x, g')} = 0 

t 


Eqn (13) 


Then it can be observed that d F (x, h) = 0 for every h (t) if 


f(x) = 2 lx (0 - d A* (x, g') = 0 (14) 


The operator fix) will be called the gradient of F(x\ 
[f(x) = grad Fix)] because it can be regarded as a generaliza¬ 
tion of the notion of gradient as commonly thought of in 
analytic geometry. 

When the gradient fix) of F(x) in the neighbourhood of a 
certain x it) = x 0 if) is known, it is possible to express the 
decrease of F(x) along the trajectory x Q (t) + oc [x ( t ) — x 0 it)] 
(where oc is changing from oc = 0 up to oc — 1) by the mean 
value 




/(*o,*) = J o 


da f {x 0 (0 + a [x (t) - x 0 (*)]} 


where \i is the probability measure, i.e. a non-negative, count¬ 
able, additive, real set function with the property fx (i2) = 1. It 
is assumed that E { A } exists and the expectation sign will be 
treated as a linear operator acting from the random variable 
space into the output signal space Y. 

Conditions of Optimality 

When x (f) is optimum, any variation yh (t) of x (t) should 
not decrease F (x). For example, taking G [x, y, yf\ if) 

_ g [y 9 y d ] one can express this condition in the form: 

dF (x, h)=-^F\_x + yK] 1 7=0 


of the gradient along this trajectory. Indeed, one obtains (see 
Remark II) the following inequality: 


\F(x 0 )-Fix)\ 

( rr ) i/<r 

< jj q |/(^x)|Mt| 



x 0 (t)-x(t)\ p dt> 


) i Ip 


(15) 


which becomes an equality when the two arguments /(x 0 , x) 
and x 0 (t) — x it) are adjoints, i.e. 


| Xo (0-x(Oi P = cons t.i/(xo^)|% p lj rq X -1 

Then from (15) it follows that the best change or variation of the 
control signal should be adjoint to the mean gradient of per- 


= 2 A 


* t f T de d , formance measure. 

x(t)h(t)dt—\ -j— \_y, y t d] "4 [x T- yh] I y = O dt Remark II. It is assumed that the weak differential d F (x, h) 

0 Joy of F (x) is a linear functional with respect to h and therefore it 

rr C T ^ can be written in the scalar product form dF(x, h) = [f(x) 9 h], 

= 2 X\xit)}iit)dt-^g f [y, y d ] dA (x, h)dt = 0 (10) where ^ xeLV [Q> n /(x) € L * [0 , T]. 


* Eqn (12): 
t Eq n (13): 


&A (x, h)=n 
dA* (x. 


fci (t~t) 

C T 

,g') = n jo 


j k 2 (t-T 1 )x(T^dTi n: k 2 ('t-x 1 )h(z 1 )dr 1 d'z 

T “|w-l pT 

k 2 (x-x Jx^OdTjJ J o k l (x l -x)g'(T 1 )dx 1 dx 


( 12 ) 

(13) 
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To prove inequality (15) let us observe that for every number 
oc e [0, 1] we have 

d r 

^T[x 0 Hha(x-x 0 )] = clF[x 0 + a(x~x 0 ) ? x-^ 0 ] 

= {f[x 0 + oc(x- x 0 )l x-x 0 } 


where x 0 (?) is an arbitrary function, provided the process 
converges, i.e. the integral distance between x n+1 and x n is 
smaller than the distance between x n and x n _ 1 


Integrating this relation we obtain: 


F(x)-F(x 0 ) = 


o 


{da/[x 0 + a (x - x 0 )], (x 0 - x)} 


Applying the Holder inequality we get the ‘maximum principle’, 
expressed by formula (15). 

The necessary condition for a minimum of F(x) can be 
written in the form grad F(x) = 6, where ||0|I = 0, 4 and for the 
sufficient condition the following formula is obtained. 


d 2 F(x,h,h)>y(\\h\\)\\h\\ 

where y (z) is a non-negative function having the property 
lim y (z) — oo. 

Z~>00 

In the case of conditional minima it must be assumed that 
the functionals are strongly differentiable or, what is equivalent 4 , 
that the weak differentials are continuous with respect to x. 
When, for instance, it is required to minimize a certain F x (x) 
subject to the condition F 2 (x) = c — const., then, for the 
necessary condition, the following equation is obtained 4 . 


grad F x (x )=X grad F 2 (x) 

where A is a number and, in addition, at point x one has 
| (grad F 2 (x)|| > 0. 

In the case when the time T should be minimized subject to 

r, e r^ traintS S n = 0. / = o, 1,..., n, in a closed sphere of 
L s F] space ||x|| < R , the problem can be solved in two 
independent steps: 

(1) Fix T and solve the conditional optimalization problem 
in an open sphere of L v [0, T], by minimizing the functional 

~ I Ml + £ o x-iFA (T), where = constant multipliers 
determined by the constraints: eW (T) = 0. 

(2) Assuming that the norm of the solution of (1) depends 
monotonically on T, the minimum T which satisfies the condition 
I \x\ | < R is found. 


Optimalizing Processes 

When A (x) is unknown one cannot solve equation (14) and 
find the best x (?) in the first interval [0, T], But it is sometimes 
possible to construct an optimalizing process x M (?), n— 0,1, 
2,... m the consecutive intervals [nT, (n + 1) T], which con¬ 
verges to the best control signal. Consider, for example, the 
problem of minimizing (8) which is equivalent to the solution of 
the equation y d (?) A (x) = 0, or the equivalent equation 

x ~ x + K '[y d (?) — A (x)] = T (x) (16) 

where k is a number. This equation can be solved by the iteration 

x n+i(t) = T [x n ], n= 0,1,... ( 17 ) 


|r(x„)-r(x„_ 1 )jMt 


i/p 


) Up 

p d t> (18) 

where < 1. 

Assuming that condition (18) is satisfied for every x (which 
sometimes can be accomplished by choosing the proper value of 
k) one can construct the sequence of functions x n (?) by applying 
x 0 (?) to the plant, observing A (x 0 ), and computing x x (?) = x 0 (?) 
d- k[y d A (x 0 )], etc. The smaller /? is, the faster this process 
converges to the best x (?). Of course, a faster converging 
optimalizing process can be constructed if one has more in¬ 
formation about the plant. When the plant changes slowly in 
time this information can be collected by observing the outputs 
y t = A (Xf) for known inputs x f and interpolating the plant 
operator by the polynomial operator (3), or equalizing the 
differentials of (3) to the differentials of the plant, determined 
experimentally. However, one cannot apply this approach in the 
case when the plant characteristics vary fast in time, because all 
the information collected in the past becomes obsolete in the 
future. Therefore it is usually better to use such iteration pro¬ 
cesses which require a minimum amount of information at every 
stage of optimalization. 

In the case when A (x) is a random operator and it is neces¬ 
sary to find the best x (?) with respect to the expected value, i.e., 
if one wants to solve the equation x = E {T(x)} = S (x), use 
can be made of an iteration scheme similar to (17) provided 
T (x) satisfies certain additional conditions. 


<fi 


0 )| 


Remark III. Namely 2 , let y be a separable Banach space and 
TyTy.. a sequence of weakly independent, weakly equally 
distributed continuous random operators mapping the Cartesian 
product Q x x into the space %, i. e.. 


ix n [co: 7] (m, x) e = JQ fi |>: 1} (co, x) e BJ , B t eB, 


B l/° : Ti (fix, x) e B] = jj. [co: T k (co, x) e 
and satisfying the conditions: 

(a) for every x ex there exists the Bochner integral: 

S 00= J (oo, x) d[i (co) 


(b) there exists a number /3 < 1 such that for every x, x, e y 
and n = 1, 2 ... *’ 2 A 


1 nyox,x 1 )-l n (co,x 2 ) 


*P||Xi-x 2 ||j = : 


A?) 


Choosing the generalized random variable V 1 arbitrarily and 
defining the mapping V n+1 -(n = 1, 2,...) of the space Q into y 
for every co e Q by the formula V n+1 (co) = S n [co, V n (co)], 
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where the mapping S n of the Cartesian product Q x % into % is 
defined by the formula 

S n (co, x)=-h X Ti(co,x) (20) 

n i = 1 

Then there exists a unique point xe%, such that S (x) = x and 
the sequence of generalized random variables converges strongly, 
almost surely (with probability one), to the fixed point x. 

The assumption (19) of this theorem can be relaxed, as was 
shown by Hans 2 , by assuming 

lx[a:\\T„((o,x)-T n (a,x)\\<p\\x-x\\] = l 
or __ __ 

||S (x)—x|| </?||x—x|| P< 1 

The equation (20) can also be substituted by: 

S„(co,x)=d-E Tj n+i ((o,x) 

K n i= i 

where k 2 , fc 2 ■••are two sequences of positive integers: 


= k,+ l,n=l,2,..., Z [l/fe n ]<°° 

i=i ■'= i 


Then each realization of the process can be used only once and 
the control x is changed less frequently the further one pro¬ 
ceeds. 

Now check whether a similar iterative approach can be 
applied in the more general case of the solution of (14). Assum¬ 
ing that 6A*(x,g r ) satisfies condition (18), one can observe 
that this can be accomplished if we can determine experimen¬ 
tally the functions cL4* [x n , g' (x„)], for every function x n (<) 
and g' (x n ). As an example consider the plant described by (11) 
for n = 1 and k 2 (/) = k 2 (t) = 0 for t < 0. The differentials (12) 
and (13) become linear adjoint operators, i.e., 


d A(x, g') = 


k(t-x)g'(x)dx, 

0 


where 


dA*(x, g')= 


k(x — t) g' (t) dr 


m= 


*t 

k t (t-x) k 2 (r)dr, 


0 


k(t) = 0 for t <0 


Theu it is easy to determine the function f* (t) — dA* [x, 
g' (t)] from /(f) = d A [x, g' (T — t)], which can be determined 
experimentally by reversing in time the input g'(T— r) and 
output f(T — t). Indeed, 


differential d A[x,g'] by reversing in time the inputs and 
outputs; i.e. /* (/) =/(T— /). In the case when k (f) is sym¬ 
metrical rather with respect to a certain time instant t jT 0 , 
than t = 0, which can be regarded as the delay (or the slope of 
phase characteristics of the linear parts of the non-linear oper¬ 
ator) one can find f* (t ) in an analogous way from the relation 
f* (f) — f (j -f T 0 — t). The same approach can also be applied 
to plants described by the sum of: 

(1) Linear, delayed by T 0 , operator: 


J* r °ic(f-To-T)x(r)dr, k(t)=0, t<0 

(2) Non-linear, delayed by T 0 , operator: (j) [x (~T 0 A t)\, 
where 0 (x) is a non-linear function. 

(3) Non-linear operator of the general type (3) which does 
not change when substituting t = T + T 0 —t, x i = T— r i9 
and interchanging the integration order. 

When not sure whether a particular plant belongs to the 
class for which the gradient can be determined by reversing 
inputs and outputs, one can test the required property experi¬ 
mentally for every input x (t), using the following criterion: 

| h 1 (f)dA [x(t),/ 2 (T)]df 


I 


T 

0 


h 2 (0 dA [x (T— t), h t (T— f)] dt 


( 22 ) 


where d A [x, h] denotes the reversed in time d A [x, h] operator 
and h 2 (f), h 2 (J) are arbitrary functions. 

It can be proved that for plants which satisfy equation (22) 
and for which the operator dA [x,g']orl/y {A [x+yg'] —A(x)} 
satisfies (18) (which can be tested experimentally) the operator 
d*A [x, g'] (which is equal to the input-output reversed in 
time d A [x, g 1 ]) will also satisfy (18), thus assuring that iteration 
processes of the type (17) will converge to the best x (f) when 
y -> 0 for n -» oo. 

A more general method of identification of dA * [x, g ] can 
be constructed using the relation 


j: 


g' (t) dA [x, K].dt = 


*r 

h(t)dA*[_x,g']dt 

0 


which connects dA [x, h ] and d^4* [x,<g']. 
Indeed, the numbers 




1 _ 

T 


% T 
0 


h l {t)dA*lx k9 g r \it, 


fe,i=1,2,... 




k(x-t) g' (t) dx 


= T k(T~t-x)g\T-t)dx=f(T-t) (21) 

J o 

In the case of non-linear operators, e. g. (11) for n> 1, a simi¬ 
lar relation holds only for certain types of non-linear operators. 
Assuming, for example, it) = k % (t) = k(t) — k ( t ) and 
/* (0 = d A* [X (r), g' (T)L /(f) = d A [x (T r),g' (T- t)] it 
can be proved that the gradient /* (f) can be obtained from the 


where h t (/) are orthogonal, i.e. 

1 f T — 0 i -=k j 

— ^ h t (t) hj (t) df = 1 j = j 

can be regarded as coefficients of the expansion of the function 
d A* [x fc , g '] into the series 


00 


cU*[x fc ,g']=Z aMt) 

i=l 
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Every coefficient can be written as 

i p 

a t = -Tp I g' ( t) d A [x k9 h t ] d t 

1 J o 

f Vm lim Sdl !t±MzAM b, 

1 Jo y-+0 t y J 

where dA [x k , can be determined experimentally. 

By assuming hit = Td (t — 1 { ) it is possible to identify 
d^4* [x Jc , g'] at any desired time moment t { . 

This method,, generally speaking, requires infinite time for 
complete determination of d,4* [x k ,g r ], k = 1, 2,... However, 
when the orthogonal functions hi (t) are properly chosen a few 
terms of a t h { (t) can provide a good approximation to d A * [x k ,g ‘']. 

When the output noise is present are random variables. 
However, it is possible to minimize the corresponding R.M. S. 
error or the so-called average risk using Bayes estimates of a it 

In this case it is also possible to improve the performance 
of the controller by collecting and utilizing all the past informa¬ 
tion about the plant characteristics: d A* [x 7c , g']. 

It should be noted that when the gradient of the performance 
measure is determined experimentally many other methods, 
such as steepest descent or Newton generalized process, can 
also be constructed and applied for the plant optimalization. 


Example 

For the sake of simplicity consider the non-random plant 
described by the operator 


H(*) = 


k(t- t)x(t) dr — e [x ( t )] 2 


(23) 


and the controller which minimizes the cost (7) for p = 2. The 
optimal iteration process corresponding to (14) is 

x n+1 (t)=~dA'* |>„(0, g'(x„)]=~dA* [x„ (t), 1 («)] 


1 

= 2l 


k (x — t) 1 (x) dt —y x n (?) X = XJX 2 (24) 


It can be shown that the plant satisfies (22) and that the process 
converges if /? = |e/2| < 1. 


d 2 F(x, h, h)=2 (X,+sX 2 ) 


h 2 (t)dt>0 if 2 1 +eA 2 >0 


Substituting x 0 (/) = 0 into (24) one gets x 1 (t) = 1/22 
dA* [0, 1 (/)]. This function can be determined by applying the 
step function y 1 ( t ) to the plant and reversing in time the re¬ 
sponse of the plant, which is multiplied by (22y) _1 . 

For the succeeding iterations we get 


and 






x (t) = lim x„ (t )= — — 

n -*• °° ^ ^ 

A 


(25) 


If e were known the best x ( t ) could be found for the first 
interval by (25). When it is not known, or is changing, one can 
still observe the jumps s/2, • x n _ x (0), at the beginning of every 
interval, and determine the value of s. This optimalizing process 
is shown in Figure 2 for the case k ( t) — oce~~ at , oc = 3/7, 
2, = 1/2, s = 1/4. It is interesting to observe that the optimalizing 
process assumes the scanning form similar to the scanning in the 
so-called extremum controllers. In the general case one cannot 
use (25) and in order to find x 2 (t) should reverse in time 1 \22 
d A [x x (T — e), 1 (t)]; a procedure which is shown in Figures 3(a) 
and (b). One applies x x ( T — t) to the plant in the first interval 
and x 1 (7—•/) +yl (t) in the third interval; then finds [see 
Figure 3(b)] 1/2 Xy {A t [x 2 (7- r) + y 1 (/)] — A m [x x (T~ t)]} 
and reverses it in time to obtain x 2 (t) etc. In order to utilize all 
the intervals we can also apply the same x x (T — r) in the even 
intervals as shown by dotted line in Figure 3(a). When it is 
observed that the initial conditions of A (x) at the beginning 
of the adjacent intervals are the same (i.e. when the steady- 
state process is obtained) [see Figure 3(c)] the step signal y 1 (/) 
can be applied and the differential dA [x x (7— t), 1 (r)] can 
be determined. 


x (t) 



A (x) 
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It should be noted that the number of intervals which are 
necessary for the determination of the differentials can be 
reduced in the case when one has two identical plants or a 
model of the plant, e.g. two chemical reactors or two or more 
cylinders of the same combustion engine which are subject to 
the same physical conditions. In this case the differential 
dA [x,„ 1] for every x n can be determined without the equaliza¬ 
tion of initial conditions. The scanning period T should be as 
short as possible, but not shorter than the settling time of the 
plant because it would not be possible to determine all the 
information contained in the transient process. It should be 
observed that in the general case of performance measure (6) 
the determination of the gradient is more complicated because 
one must reverse in time also g' (x n ) and in many cases should 
predict the desired state y d . 

It is interesting to compare the value of the performance 
measure eqn (7) for the optimum control figure eqn (25) and the 
corresponding value obtained when a conventional extremum¬ 
seeking regulator is being applied. Such a regulator ignores 
usually the input costs and transient processes in the linear par 
of the plant and generates an input x k which maximizes the 
steady-state output of the plant. Assuming very small hunting 
amplitude, generated by this controller, we obtain the following 
best input 

1 

Xk ~2e’ 

and the corresponding value of the performance measure is 


F(x*) = 4^1-84) 

Computing the corresponding value of the performance 
measure F (x), when the periodic signal eqn (25) is being applied, 
one gets 


a ^2/d ~-2ctT\ a ^2 T 


Foo^a-e 


T )- 


a—- 


2 5 2(A 1 + fiA 2 ) 

For a sufficiently long T (e.g. <xT > 3), we obtain 

X\T 


F(x)-F(x k )*- 


4 (Ai + &X 2 ) s' 


r < 0 


Because > 0, the optimum controller which gen¬ 

erates eqn (25) is ever better than a conventional extremum- 
seeking regulator. 
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Non-Linear Programming in the Investigation of Optimal 

Automatic Control Systems 

N. I. ANDREEV 


Summary 


Non-linear programming problems may be met with in the selection 
of optimal automatic control systems. 

This paper presents a method of solving a problem in non-linear 
programming. The essence of the method consists in reducing the set 
problem to a repeated search for a solution of a linear programming 
problem and the choice of values for certain additional parameters 
that are introduced. 

Sommaire 

Des problemes de programmation non-lineaire se rencontrent dans 
1 etude de systemes d’optimalisation automatique. Ce rapport presente 
une methode pour la solution de problemes de programmation non- 
lineaire. Le principe de cette mdthode consiste k ramener le probleme 
pos6 a une recherche par iteration de la solution d’un probleme de 
programmation lineaire et au choix de la valeur de certains para- 
metres supplementaires a introduce dans les calculs. 

Zusammenfassung 


automatic control systems. It should be noted that in a number 
of practically important cases the investigation of automatic 
control systems reduces to a complex problem—a non-linear 
programming problem, whose solution has so far only been 
obtained for certain particular cases 4 . 

This paper puts forward a method of non-linear programming 
suitable for the solution of a broad range of problems. This 
method relies essentially on the techniques of linear programming. 
Therefore the formulation of the linear programming problem 
is set out below. 

As is known 4 - 6 , this problem is expressed in the following 
manner. It is necessary to find the greatest value of a linear 
function of n variables x u x. 2 ,..x n 

L = L{x 1 ,x 2 , ...,x n ) = p 1 x l +p 2 x 2 + ... +p„x n (1) 
when the variables are subject to constraints of the form 


Die Synthese von Optimalwertregelungen fiihrt auf Probleme der 
mchthnearen Programmierung. Der Aufsatz enthalt eine Methode zur 
Losung ernes solchen Problems der nichtlinearen Programmierung. 
Im wesentlichen besteht die Methode darin, das gegebene Problem auf 
ein wiederholtes Suchen nach einer Losung eines Problems der linearen 
Programmierung zu reduzieren und in der Wahl der Werte von be- 
stimmten zusatzlich eingefiihrten Parameter!!. 


a n x i+ ■■■ +a„i x „ = b 1 

a lm x l+'--. +a„m x n—b m . 

du x i+...+d nl x n <l 1 ] 


( 2 ) 


This paper presents a method of solving a problem in non-linear 
programming. The essence of the method consists in reducing 
the set problem to a repeated search for a solution of a linear 
programming problem and the choice of values for certain 
additional parameters that are introduced. Non-linear program¬ 
ming problems of a similar nature may be met with in the 
selection of optimal automatic control systems. 


dlr x l + ... + d nr x n < l r 


(3) 


The relations (2) and (3) determine the region C of variation 
of the variables %,..., x n . These conditions can be transformed 
m such a way that either m or r becomes zero 4 . In actual problems 
one uses the method of writing the conditions that is most 
convenient. 


Presently linear programming has deeply penetrated into the 
techniques used for investigating automatic control systems 
Academician Pontryagin’s method 1 , which determines the 
optimal control for an automatic system in a number of practi¬ 
cally important cases (e. g. the solution of the problem of optimal 
linear high-speed action), contains a linear programming 
problem as one of its intermediate stages. Bellman’s method of 
dynamic programming’, which is of great generality and is also 
used for investigating automatic control systems, has a linear 

Srr™ ^ “ intermediate ^age in a number of 
selected ! P ? f l t funCti0n is Iineady dependent on the 

for - para , m ? t< ; rs) - Lmear Programming methods are used 
for solving reliabdity problems 3 , problems of rational tolerances 
m the production of assemblies 4 , and many other problems 
ose y connected with the investigation and development of 


“ LUC iuncuon jl serves as an index of the 

quality of the solution. The parameters x h ..., x n are character- 
istic of the object and the investigation, and have various 
physical significances according to the problem. For example 
m solving a problem on high-speed action these parameters 
appear as control actions. 

A geometrical interpretation can be given to the linear 
programming problem as follows: it is required to find the 
greatest value of linear function L of the variables x, ... x 
whose variation is confined to a region G given in the form of a 
polyhedron in n-dimensional space. 

Efficient techniques have been developed for solving the 

m P tbnd Pr0Sramn ?“i, Pr0b J em4 ' But lhe linear Programming 
method is inapplicable when either the quality index is a non¬ 
linear function F(x lt ..., x n ) or the region G of variation of the 
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parameters x lt .. x n is determined by non-linear relations 
between them. Such cases arise, for example, in solving the 
high-speed action of a system: 

1. If the equations of the system include non-linear terms 
in the control parameters (quality index a non-linear function 
of the parameters); 

2. If the region G of variation of the parameters is deter¬ 
mined by non-linear relations, e.g. of the form 

x 2 x + ... + x 2 < R 2 

(the region of control forms a hypersphere centred on the 
origin). 

If only the relations defining the region G are non-linear 
while the quality index is a linear function, one can replace this 
region by one bounded by relations of the same type as (2) and (3) 
which coincide accurately enough with the original region 
(e. g. the hypersphere may be replaced by a polyhedron circum¬ 
scribed to it). The problem is thus reduced to one of linear 
programming. 

If the quality index is a non-linear function F while the region 
G is determined by linear relations such as (2) and (3), one can 
sometimes replace the non-linear function F(x l9 ..., x n ) by one 
that is piecewise linear, and proceed to solve the problem by 
linear programming methods 4 . But this device cannot always be 
used, and involves very bulky computation when it is applicable. 

In view of what has been said, the following formulation 
of the non-linear programming problem is of practical and 
theoretical interest. Let the quality index be a given non-linear 
function F of the variables x l9 .. x n . Without loss of generality, 
it may be considered that the function F(x 3 , x n ) may be 
represented as a function ® of certain linear forms L l9 L 2 ,..., 

ifc+1 F(x 1) x 2 ,...,x„)=1>(L 1) L 2 . L k+x ) (4) 

where ® is a given function of the variables X 3 ,..., L ]c +i\ 

+ ••• + Qin x *i 

the q i3 - being given numbers for / = 1, 2,..., n and ; = 0, 1, 2, 

..7z, while k < n. 

It is required to find the greatest value of the function F 
under the conditions (2) and (3). 

Before proceeding with the solution of this problem, it must 
be explained why the function F is replaced by 0. The fact is 
that in many practical problems the number n of variables is 
large, and this severely complicates the process of finding a 
solution. Therefore it is worth while, if at all possible, to go 
over from the function F of many variables to the function ® 
depending on a lesser number of variables Li. Such a transition, 
as will be seen from what follows, simplifies the procedure for 
obtaining a solution. 

Two examples are given to illustrate this method of transition 
to a smaller number of variables. 


Example 1—F(x lt x 2 , x z ) = x\ + x\ + A + 2 x 2 x 3 . 

This function of three variables x l9 x 2 and x 3 can be 
expressed as a function of two other variables L x and X 2 : 

Lj + Lj 

F(x 1 ,x 2 ,x 3 ) = 0(L 1 ,L 2 )=—£— 

where L x = H- * 2 + L 2 ~x 1 — x 2 — x 3* 

Here n = 3 and k = 1. 


Example 2 —F (x x , x 2 ) = x\ + x \. 

This function of two variables cannot be expressed as a 
function of a lesser number of variables L { . In this example one 
may put Z 3 — x x and Z 2 = x 2 . Here n = 2 and k = 1. 

The greatest value of the function F in the region G of varia¬ 
tion of the variables x l9 ...,x n as defined by conditions (2) and (3) 
coincides with the greatest value of the function® in the region Q 
of variation of the variables L t , ..., L k+1 as determined also in 
the final analysis by conditions (2) and (3). The greatest value 
of ® may be attained either within the region Q or on its 
boundary S. Consider each of these cases separately. 

First Case 

Suppose the function ® attains a maximum within the 
region Q. In this event the problem reduces to finding a maximum 
of a function of k + 1 variables. It is known that a necessary 
condition for® to have a maximum is that its partial derivatives 
should vanish: 

j£=0, i=l,2,...,/c + l (6) 

o L t 

at a certain point in the region Q of the parameter space 
(Li, ..., F/c+i). 

If ® is not differentiable everywhere inside Q, some of the 
conditions (6) may be replaced by these: 

d®/hL v does not exist, = 1, 2, ..., m <k l. 

In the general case the system of eqn (6) may have several 
solutions. Out of them must be chosen the one that corresponds 
to the greatest value of®. Suppose this solution has been found: 

Li=L im , i=l,2,...,k+l (7) 

Substituting the values (7) of the variables Li in eqn (5), it is 
possible to determine the values of the quantities Xj — Xj m at 
which the required greatest value of F is attained. Thus, in this 
case, the problem is solved by using the normal methods of 
classical analysis. Conditions (2) and (3) are here used only to 
reject those maxima ofd> (or F) that do not fall within Q (or G ). 
This first case is rarely met in practice, since the quality index 
is normally taken as a function F which has no maximum within 
the region G. The case considered below is of greater practical 
interest. 

Second Case 

Suppose the function <£> has no maximum within the region Q, 
and attains its greatest value on the boundary S of this region. 
In this case the determination of the greatest value cannot be 
solved by the techniques of classical analysis, and so the follow¬ 
ing two-stage method is proposed for solving this problem. 

In the first stage one must determine the boundary S of the 
region Q, while in the second, one finds the greatest value of the 
function 4) on S. Here one may make use of the ideas and 
techniques developed by the author 7 - 8 , applying them to a 
problem of a different nature. 

To determine the boundary S one may proceed in the 
following manner. For fixed values of the variables 

L i = C u L 2 =C 2 ,...,L k =C k (8) 

one must find the greatest (and least) value of L k+1 (see Figure 1, 
where k = 1). 
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Since the greatest and least values of Z, 7c+1 are determined 
by similar means, from now on only the greatest values of 
Lj-'n are mentioned (i.e. only one half-branch of 5 is dealt with). 

It follows that, to find one point on S, one must obtain the 
greatest value of the linear form L k+1 under conditions (2), (3) 
and (8). This is a typical linear programming problem. Condi¬ 
tions (8) have essentially changed nothing in conditions (2) 
and (3); the number of equations has merely increased by k. 
Taking various values of the parameters C l5 C 2 ,..., C k , one 
can also derive the points on S corresponding to them. If these 
points are chosen so as to cover the whole of S densely enough, 
the first stage of the problem may be considered solved. 



The partial derivatives bf/bQ may be derived analytically if one 
has succeeded in obtaining a simple analytical expression for f 
But one cannot count on this, since normally the expression 
for / is complicated and, what is more, cannot be derived in 
explicit form. Thus in the general case the derivatives bf/bQ 
must be obtained approximately as the ratio of finite differences 


0C* A Q 

where Ay;. =/(C U) C^\, C n + AQ, C( i+1 ) 1; C kl ) - 

-/(C n ,...,C M ). 

After determining the gradient of/at the point (C n ,..., C kl ), 
a displacement in the space G is made along this gradient vector, 
i. e. the values of / are considered for the following values of the 
variables: 


Cl Cll+ SC 1 ‘ e ’ C2_C21+ ^‘ £ ’ C * = Cfcl+ J^' £ 

where the bf/bQ (/ = 1,2,..., k) are evaluated at Q - C n , 

Q = Qi» • ••> c k — c kl . 

The displacement in the chosen direction is terminated at 
the value e = e 1 at which the function 


Now it is necessary to solve the second stage of the problem, 
i.e. to find the greatest value of $ on S. This is easily solved if 
the number k of dimensions of 5 is small. In this case the 
greatest value of €> can be determined approximately by com¬ 
paring the values of tP at the nodes of a network formed by 
discrete values of the numbers C lf C 2 ,..., C k . If the number k 
of dimensions of S is large, producing a close network of values 
of O on S becomes an extremely laborious task, which cannot 
always be carried out in a reasonable time even by the use of 
modem high-speed computer techniques. 

In this case the determination of the greatest value of <E> 
reduces to finding the maximum of the function 

/=/(C 1) C 2! ...,Q) = 3>[C l5 ... J C i ,L t+1 (C 1> ...,Q] (9) 

where Aj. +1 (Q,.. C k ) is the greatest (least) value of the linear 
form L k+1 under conditions (2), (3) and (8). The greatest value 
of/ on 5" in general coincides with the maximum of this function, 
i. e. is attained within the region of variation of the parameters Q. 

To determine the maximum of / = /(C ls C k ) use can 
be made of the method of most rapid descent 8 . 9 . The combina¬ 
tion of the method given above (which leads to the boundary S 
of the region Q, and to the function/) and the method of most 
rapid descent (leading to the maximum off) makes it possible 
to avoid the computation of values of/at a large number of 
points densely covering the whole region S of variation of the 
variables C lt ..C k , and to replace these bulky calculations by 
more economic ones according to the following plan. 

Let a first approximation to the variables 

< ~’i = CiuC2 = C 21 ,C k — C kl 

be chosen from any considerations. To this corresponds a value 
of the function/ L =/(C u , C 21 ,..., C kl ). Now the direction of 
the gradient of/at this point is determined, which as is known 
is given by a vector in the space G = (Q,..., C k ), whose 
projections on the C 1( C 2 , C k axes are respectively 

jy 

acy acysc* 


reaches a maximum. This maximum of (s) may be determined 
graphically (see Figure 2). 

The values of the variables 


c i - C x2 - C 1 ! +—e x ,.. •, C k = C k2 = C k ! s t 

are taken as the second approximation. The value of the function 

/ = /2 ~f(C 12 ,..., C A2 ) is taken as the second approximation 
to/. 

Then the third and succeeding approximations to the 
variables C l9 ..., C k and the function / are obtained by the 
method given above. 



The (v -f- l)th approximation is given by 

r Of Qf 

1 (»+!)— lv + dC 1 £v ’ ^(^+ 1 ) = ^ kv + QC’ me v 

where e = e v corresponds to a maximum of the function 

. C kB +~-[ 

fv+i=f (c 1(v+1 ), c 2( „ +1) , ...,c k(v+i) ) 

The process of finding the maximum of/is terminated when two 
successive approximations to/differ by a negligible amount. 
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In the general case the function / may have several maxima, 
and it is necessary to find the greatest of these. It should be 
noted that in the general case the greatest value of/is attained 
within the region S of variation of the parameters C l9 C 2 , ..C fc , 
i.e. it coincides with a maximum of this function. Only in rare 
individual cases is the greatest value of/attained on the boundary 
of the region 5. This assertion follows from the fact that in the 
general case the function F(x l9 x& ..., x n ) reaches its greatest 
value on a face, and not at a vertex, of the polyhedron defined 
by conditions (2) and (3). 

Based on the above, the following sequence of operations 
can now be recommended for determining a maximum of the 
function /. 

(a) Choose the first approximation to the variables 
C'i = C n , ..., C k = C kx . 

(, h ) Compute the value of the first approximation to the 
function / = Z = /(C u , ..., C kl ). 

(c) Evaluate the components of the gradient vector of / at 
the first approximation point: 

J L 

S Ci'" 9 dC k 

(d) Calculate the function 

«l(8)=/(c u +^-8,...,C M +^8) 

for increasing values of the parameter e — Ae • /, where 
/ = 1, 2, ... . The increment Ae is chosen in accordance with 
the peculiarities of / that become evident during the process of 
computation: the more gentle the variation in /, the larger 
can Ae be taken. 

(e) Determine the value of the parameter e = e x that makes 
the function (e) a maximum. 

(/) Determine the second approximation 

C*12 = ^*11 "b &19 • * CfrZ — Cfcl "b 

(g) Evaluate the second approximation to /: 

/j — f (C] i2, C 22 j • • •> C; c2 )* 

(h) Calculate the difference between the two successive 
approximations to / i.e./ 2 — /. 

This sequence is continued until the difference 

/+:i - / =/(C a ( <+1 ),..., C 7c ( i+1 )) - f(C lt , ..., C 7ci ) 

becomes negligibly small. Ordinarily the number of approxima¬ 
tions that have to be taken when using this technique is not great. 
The computations involved can readily be programmed for a 
computer dealing with finite differences. 

One may naturally wonder whether the method of most 
rapid descent cannot be applied directly to determining the 
greatest value of the function F(x l9 ..., x n ) under conditions (2) 
and (3). In principle, this approach is also possible, but it leads 
to substantially more complex calculations in the cases where (a) 
the number n of variables x l9 . . x n is significantly greater than 
the number k of variables C l9 ..., C k9 and (b) the number of 
inequalities (and equations) in conditions (2) and (3) is large. 

The considerable increase in the volume of computation in 
the first case needs no explanation. In the second case, it arises 
from the fact that the direct application of the method of most 


rapid descent here requires that at each step of the calculation, 
when e is increased by Ae, one has also to check whether or not 
conditions (2) and (3) are satisfied. Also the transition from one 
face to another of the polyhedron defined by (2) and (3) involves 
a change in the form of a function of n — r variables. 

This complicates the programming of the computation. 
It follows that the volume of work in deriving each approxima¬ 
tion increases, and so does the number of approximations. 

When the number of inequalities in (3) is small and 
k -f 1 = n 9 both the methods become roughly equal in 
time-consumption. 

These two different cases have been considered above: 

(a) The greatest value of F (and <3» is attained within the region 
of variation of the variables x l9 ... 9 x n (or L l9 ...,L k+1 ), and 

(b) the greatest value of F (and <D) is attained on the boundary 
of this region, the function having no maximum within the 
region G (or Q). 

The case may arise [although also improbable, as (a) above] 
where the function F (or <D) has a maximum within the region G 
(or Q), but attains its greatest value on the boundary of this 
region. Consequently in this case the maximum of <E> has to be 
found and compared with the greatest value of this function 
reached on the boundary S 9 and the greater of the two has to 
be chosen. 

It may be expected that the techniques of solving non-linear 
programming problems will develop in the future, and that 
experience in this field will accumulate. Therefore it is worth 
making the following more general statement of the problem. 

Let there be a method for determining the greatest (and least) 
value of the function T (x l9 ..., x n ) under conditions (2) and (3) 
that are imposed on the region of variation of the variables 
x l9 ..., x w . It is necessary to find the greatest value of the function 

F(x u ...,x„) = ( S>( x ¥,L 1 ,...,L k ) (10) 

where F p — -i - (jp X x x ~b ... Qpn Xn, p ~ 1, 2, .. k 9 k ti 9 

and conditions (2) and (3) are satisfied. 

Consider 0 as a function of the k + 1 parameters T, L l9 
..., L 7c . The greatest value of this function may be attained 
either within the region Q of variation of these variables or on 
its boundary. 

If the greatest value of <E> is reached within Q (an improbable 
case in practice), then the problem reduces to finding the maxima 
of this function, which are determined by the equations: 




17 = 1,2, ...,/c 


( 11 ) 


These equations enable one to determine the values of the 
functions T* - T 0 , L x = L m ...,L k - L k0 , which correspond 
to a maximum of ®. If the solution of eqn (11) is not unique, 
then one must choose from all its solutions the one that corre¬ 
sponds to the greatest of the maxima of <E>. From the relations 


x F(x 1 ,... J x„) = v F 0 

<2pO + <i/l ;X: l+ *“ -^qpnXn — LpO, p=l,2, ...,/C (12) 

one determines the values of the ^variables XlQ> -^20> • * -5 
corresponding to the greatest value of the function <E>. In the 
general case the solution of the system (12) is not unique. 

If, however, the greatest value of ® is reached on the bound¬ 
ary S of the region Q (which is more likely in practical cases), 
then it is desirable to solve the problem as stated in two stages. 
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First, one must find the boundary S , and then determine the 
greatest value of ® on it. In determining the boundary S , it is 
necessary to take given values of the linear forms 

l 1 =c 1 ,l 2 =c 2) ...,4=c* (13) 

and then determine the greatest and least values of the function 
Y (x 1; x 2 ,..., x n ) under conditions (2), (3) and (13). 

It has been pointed out above that there is a method for 
solving this problem [the addition of (13) does not in principle 
alter conditions (2) and (3)]. Taking various given values of the 
parameters C x ,..., C 1: one may obtain the corresponding values: 

'f'l = 'F(C 1 ,C 2 ,...,C 4 ) 

* =® DP (C u c k ), c Jf .... CJ=/(C 1S .... c k ) 

x i = x i (Pu C 2 > ***5 Ck) 1, 2, ..., n 

The second stage of the solution consists in the determination 
of the maximum of / = /(C, C k ) and the values of the 
variables 

*1 ~ X 10j x 2~ x 20> •••> X n ==X nO 

corresponding to this maximum. This part of the solution is 
carried out in exactly the same way as for the first statement of 
the problem. A simple example is now given to explain the 
technique that has been proposed for solving the non-linear 
programming problem. 


and least values of L 2 . The question arises of how to choose 
these given values of L x . This question is easily answered. The 
greatest and least values of L x under the above conditions are 
readily obtained by linear programming methods and are: 

0<L X <30 

Taking a certain value L x = C l5 where 0 < C x < 30, the greatest 
value of L 2 is found (the least value of L 2 is of no interest in 
this example, since <I> is a monotone increasing function in the 
variable L 2 ). This greatest value is easily obtained by linear 
programming methods (or by other means), and may be expressed 
in terms of C x in the following form: 

L 2 ~60 — 2C X 

The function d> can be expressed in terms of the parameter C x 
as follows over the section of S that is being considered: 

®=C X (60 — 2 Cj) 

It can readily be seen that the function <E> on the boundary S 
attains a maximum at C x = C 10 = 15. 

Now it is easy to determine all the quantities of interest: 

0>o = F 0 = 15 (60 -2-15) = 450 

x lo = 60 — 2-15 = 30 

The values of x 20 and x 30 are obtained from the equations: 
x 2 + x 3 = 15 


Example 

Determination of the greatest value of the function 

F (xj, x 2 , x 3 )=x^ (x 2 +x 3 ) 
under the conditions: 

x 1 -j- 2 x 2 + 3 x 3 < 60 

x i>0, x 2 >0, x 3 >0 

The given function F of the three variables x l9 x 2 and x 3 can 
be expressed as a function of two linear forms L ± and L 2 : 

L 1 =x 2 + x 3 , L 2 ~x 1 

with 

® F 2 )=L 1 ' L 2 

In this example O is a monotone increasing function. Hence 
it has no maximum, and attains its greatest value on the 
boundary S. Following the procedure set out above, one deter¬ 
mines the boundary S of the region Q of variation of the linear 
forms 1^ and I 2 . In this case the boundary, is a certain curve 
(a one-dimensional domain). 

In order to find $, it is necessary to take various given 
values of the linear form L x and to evaluate for each the greatest 


30 + 2 x 2 + 3 x 3 = 60 (see above conditions) 

Solution of these equations gives the following values for x>n 
and x 30 : 20 

x 20 = 15, x 3O =0 
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Numerical Analysis of Non-linear Control Systems using 
the Fokker-Planck-Kolmogorov Equation 

K. J. MERKLINGER 


Summary 

The Fokker-Planck-Kolmogorov equation is a means of describing 
the response of non-linear control systems to Gaussian signals and 
disturbances. A method for numerical solution of the equation is 
presented here. The method is applied to first, second and third order 
examples of relay control systems with simple gaussian inputs. The 
error distribution is computed by marginal integration of the state 
vector distribution, and the mean squared error is used to test the 
advantage of linear or parabolic switching functions, and the validity 
of Booton’s linear approximation. 

Sommaire 

L’equation de Fokker-Planck-Kolmogorov est un moyen de decrire 
la reponse de systemes de commande non lineaires a des signaux et per¬ 
turbation gaussiens. Le rapport presente une methode de resolution 
numerique de cette equation. La methode s’applique a des exemples 
du premier, du deuxieme et du troisieme ordre de systemes de commande 
a relais avec entrees gaussiennes simples. La distribution de l’erreur 
est calculee par integration marginale de la distribution du vecteur 
d’etat, et l’erreur quadratique moyenne est utilisee pour essayer l’avan- 
tage compare des fonctions de commutation lineaires ou paraboliques, 
et la validite de Papproximation lineaire de Booton. 

Zusammenfassung 

Mit Hilfe der Gleichung von Fokker, Planck und Kolmogorov kann 
man die Systemantwort nichtlinearer Regelsysteme auf Zufallssignale 
und Storungen mit GauBscher Verteilung (GauBsche Signale) be- 
schreiben. Ein Verfahren fiir die numerische Losung dieser Gleichung 
wird angegeben. Es wird auf Relais-Regelsysteme erster, zweiter und 
dritter Ordnung mit einfachen GauBschen EingangsgroBen angewen- 
det. Die Fehlerverteilung wird durch Randintegration fiber die Ver¬ 
teilung des Zustandsvektors berechnet; der mittlere quadratische 
Fehler dient der Prufung der Vorteile der linearen und der paraboli- 
schen Schaltfunktionen und zum Nachweis der Gilltigkeit der linearen 
Naherung nach Booton . o ■ , • 


Introduction 

The equation of Fokker, Planck and Kolmogorov is known 
by famous works in probability and physics, and by the 
paper of Barrett at the First IFAC Congress. The edhnection 
between the distribution of a Brownian particle arid a partial 
differential equation was discovered by Einstein 1 , and developed 
by Smoluchowski, Fokker and Planck. More general equations 
for Markov processes were presented by Kolmogorov 2 , and 
applied to the study of dynamical systems by Andronov, 
Pontryagin and Witt 3 . 

In automatic control, the equation has been demonstrated 
by Chuang and Kazda 5 , Pugachev 6 , Barrett 7 and Florentin 8 , as a 
means of describing the response of non-linear control systems to 
Gaussian signals and disturbances. Unfortunately, the equation 


has been found largely intractable, except in special cases such 
as linear systems for which a general solution has been given by 
Wang and Uhlenbeck 4 . The first examples from Brownian 
motion studies were linear. Non-linear examples have been 
solved by Fuller 9 , Khazen 10 and Sawaragi et al. 11 , by piecing 
together elementary solutions for the linear regions of phase 
space, and satisfying some conditions along the boundary of 
each region, but it is not generally possible for elementary 
functions to satisfy the boundary conditions. 

This paper presents a finite difference method by which 
approximate numerical solutions are obtained. The method is 
applied to first, second and third-order examples of relay control 
systems with simple Gaussian inputs. The accuracy of the 
approximate solution is tested by comparison with analytic 
results in two cases where they are available. The error distribu¬ 
tion and mean squared error are obtained. Linear and parabolic 
switching functions are compared, and the accuracy of some 
results from statistical linearization is tested. 

The Fokker-Planck-Kolmogorov Equation 

The canonical equation of motion of a control system v^ith 
stationary Gaussian inputs is written 

z=nz)+a (i) 

where the vector z represents the combined state of inputs and 
plant, and f is a Gaussian white noise vector independent of 
z and 1 . The components of z which represent an input x, 
where x is an nth order Gaussian process, may be x and its first 
n-1 derivatives. 

The Fokker-Planck-Kolmogorov equation is derived by 
Kolmogorov 2 , Barrett 7 , and others from the transition, law for 
Markov processes, and the equation of motion (1). It may 
be written 

8(0 (z,q g oto) = - V .j (z , f)= —x|-j t(z ,0 (2) 

where oj (z, 1 1 z 0 f 0 ) is the conditional probability density 
function of the state vector z, and / (z, t) is a vector given by 

ji (z, 0 =fi(z) co(z,t\ z 0 t 0 ) - -by >7i* ( z > 1 1 Z 0 t 0 ) (3) 

The functions f t are obtained from the equation of motion (1), 
and the constants rj ik are the cross spectral densities of the 
Gaussian noises and 

The Fokker-Planck-Kolomgorov equation describes a flow 
of probability in phase space. Probability particles diffuse under 
the influence of the Gaussian noise f, and drift (towards the 
origin if the system is stable) under the influence of the control 
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forces implicit in f. The equation is an expresssion of the con¬ 
tinuity of the flow vector /. 

The required solution is subject to the initial condition 


limco(z, 1 1 z 0 t 0 ) = 5(z — z 0 ) 

t~+to 

and the normalizing condition 

<o(z,t\ z 0 t 0 ) dzj... dz„ = 1 


/■ 


(4) 


(5) 


where R is the whole phase space. Of particular interest in 
control system design is the case where co converges with time 
to a stationary distribution independent of the initial condition, 
since statistical parameters of the response over a long period 
of time may be evaluated from the stationary distribution. In 
this case. 


and we shall write 


dco 

hm--=0 

8 1 

(6) 

lim co(z,t\z 0 t 0 ) = co (z) 

f-> 00 

(7) 


and obtain from (2) the stationary equation 

V-/(z)= -^{/,(z)<»W}4x(8) 


Boundary Conditions 

It is apparent that eqn (8) does not exist unless the functions 
fi and co are differentiable with respect to the z variables. In the 
case of relay control systems, the drift vector / is not differen¬ 
tiable on the switching surface C. Boundary conditions are then 
applied along C which effectively piece together solutions for 
the two regions bordering upon C, in which / and co are assumed 
to possess sufficient derivatives. Suitable boundary conditions are 

(a) J - n is continuous, where n is a unit vector normal to C, 
and 

( b ) co is continuous. 

The necessity of the first condition is made apparent by the 
application of Gauss’ Theorem to a thin volume V, parallel 
with and containing C (Figure 1). We can write 



Figure 1. A thin volume containing the switching surface C 




V-J dV= 



(9) 


where S is the surface of V 9 and n is a unit vector normal to S. 
Since in the stationary case, b co/b t = 0, one obtains 


J-ndS = 0 


( 10 ) 


In the presence of the diffusing vector f, it is reasonable to 
postulate that no singularity of particles exists on surface C. 
Then, if V is infinitesimally thin, there is no measurable flow of 
particles through the sides of V, and one can write 


fj-ncLS = 


J-ndS + 

«. s 

' S + * 

J s- „ 


J-ndS = 0 (11) 


where S + and S are the two faces of V. In the limit, when S + 
and S~ coincide, one obtains 


J + -n c = J -n c 

where J + and J~ are flow vectors on opposite sides of C, and n c 
is a unit vector normal to C. 

The second boundary condition (co is continuous) is postu¬ 
lated on the grounds that any discontinuity in the density of 
particles will be dissolved as quickly as it occurs by the diffusive 
force of £. 


Finite Difference Methods 


Characteristically, finite difference methods divide the domain 
R of the z variables into rectangular cells. The quantization may 
be accomplished physically or mathematically. In the physical 
interpretation, eqn (8) is replaced by laws for the transition of 
particles between cells, given the transition probabilities of a 
vector Markov chain with cells as states. Applied to all cells, 
the transition laws generate a system of difference equations. 
The same system of equations may be obtained by replacing the 
derivatives in eqn (8) with difference formulae analogous to the 
transition laws. 

The following discussion will be restricted for simplicity, to 
the two-dimensional space R 2 = (z ls z 2 ), Consider a network 
of lines to divide i? 2 into rectangles, and let W(z l9 z 2 ) denote 
a function in R%, The value of W at the nodes of the network 
will be denoted by 

W (z l9 z 2 ) = W ( mh , nk ) = W mn 

where m and n are the integers 0, ± 1 , + 2,. .., and h and k are 
the dimensions of the rectangles. The nodal function W mn will 
satisfy a difference equation 


L hk (WJ) = 0 

with the desired convergence property: 


lim L hk (W mn ) = L(co)=-Y J ^fi(z)co(z) 

h,k->0 / ~ w 


+ r?f^0^ co(z) 


( 12 ) 


(13) 
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gp? 


It is often hoped that this property is sufficient to provide the 
further convergence, 

lim W = co (14) 

h,k-+ 0 

That it is not sufficient as an important negative statement from 
the finite difference theory (given, for example, by Forsythe 
and Wasow 12 ). 

An operator will be required which possesses the con¬ 
vergence property (13) and which, in addition, yields a solution 
for L hk (W mn ) = 0, such that 


I l a m,n I I l a m,n - u l | 

l2F +— 4Fl Wm+l ’ n+ \Ti? + ~Tfr\ 


+< ' _ F + T bm ’"-“ 2 + 2 c ^"- u}Wm 


Hj n + 1, n — u 


+<! -p 2 Cm ’ n -“ 1 \ Wm ’ n - u 


(IB) 


| a>(z lt z 2 ) - W(z u z 2 )| is sufficiently small (15) 

where W mn has been extended by interpolation to cover R 2 . 
Further, one would like to obtain L hk and solve (12) in the most 
economical way. 

Unfortunately, it can rarely be determined from theory 
whether (15) is satisfied, and one must adopt an experimental 
approach such as testing a difference procedure for some 
problems whose exact solutions are known, or testing the 
procedure for various values of h and k and observing the 
stability of W. 


A Finite Difference Model 

The Crank-Nicolson Formula 

Consider the control systems having only one input, for 
which the stationary equation in two dimensions may be written 


i « -i 

+ \IF 4F~I 


w 

rr m 


-1,H 


+ 


1 

127? 


4 h 


i 

| Urn - 1, it — ti 


= 0; m, n = 0, ±1, ±2, ... 


Non-uniform Cell Sizes 

Since the required function W is expected to be smoothest 
away from the origin (if the control system is good), it will be 
expedient, for economy of points, to introduce the transforma¬ 
tion 

(zi,Z2)-Kzi> z 2) ^ ^ 

where 

Zj->e Zi —1, z ; >0 

and , 

z t -* 1—e z ‘, z,<0; i = l,2 




dzj 


- + fl(z 1 z 2 )—- tuwi; 0Z2 

+ c(z 1 z 2 )co(z 1 z 2 ) = 0 (16) 

The derivatives in the above equation will be replaced by the 
Crank-Nicolson difference approximation 13 as follows: 


^ T \i{W m+ x,n-2W mn +W m - hn 

dZl +W m+1 , n - u -2W m , n - u +W m - Un - u } 

OZ j 4 n 

(17) 

— > - (w —w ) 

6z 2 k {¥Vm ’ n 

co^{W m , n + W m , n - u } 

where u =sgn C (z lf z 2 ) 

and C(z l9 z 2 ) = 0 is the switching curve. 

The arguments of the coefficients a , b , c will be interpreted as 
(m h, (i n — m/2) k). 

Experiments with eqn(16) have shown that the Crank- 
Nicolson formula is more accurate than simpler asymmetric 
formulae, and the symmetric formula of Richardson 12 is unstable. 

From the Crank-Nicolson formula we obtain the system of 
difference equations given by 


The transformation (19) is equivalent to dividing the (z 1; z 2 ) 
space into non-uniform cells such that the nodes (m, n )i are 
denser towards the origin. The system of difference equations 
for W ( Zl \ z 2 ') = W (m h, n k) = W mn may be written 


+ 


+ 


1 — 2 m/i _ _ 

27? 4/i 


1 — 2 mh _ 2 

2/? 6 4 h 




a' u 

u m, n - 2 


-2mh -m/f / M 

e ~e a ™> n - z 


+ 1 ,n 

w m+1 ,n + n 


_J,e— 


( 20 ) 


, . 1 -2mh U A ~nk 

+ ->-p e 


u -nku' u J__ / 




+ 


4- 


1 — 2 m/i ._L 

2F 4h 


1 — 2 m/t ■ j_ 

2F e 4/Z 


— 2 mh -mh 

e C 


— 2 mh f^-mh 

e c 




“1 - !* 


= 0; m, n = 0, 4-1, 4-2, ... 
and 
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/ ^ +2mh ._ 

\lh 2t + 4 h 


t 2 mh , 1 o, + 2 mh . mh _ 

T 77 e +e a n 


w m+1 , n 


1 „2m/i , 1 


+ ( 2 F el "‘ + 4 TL e 2 "‘ +e " < '".-:Jf l, '»-...- 


-P 2m/l _ ^ &~ nk h , 1 ( TX r 

2 A: +2 j 


j_ J _JL_ e 2 _L 

+ 2/, 2e 4fc 


I 

~ u m,n~ ( 


—I— J ^ P -i *2 mh . tm/i 

[2/! 2 4/T|_ +e a ».»-=J| ^-1.--. 

=0; m = 0, -1, -2, ..., n =0, +1, +2, .... etc. 

Boundary Conditions at the Switching Curve 

It has been stated, in the analytic discussion, that eqn (8) 
does not exist on the switching curve C (z l5 z 2 ) = 0, and that 
boundary conditions are applied along C which effectively piece 
together solutions for the regions C (z v z 2 )> 0 and C(z,, z 2 ) < 0. 
Similarly, the difference eqn (20) may not be written when 
pomts (m + 1, ri) and (m — 1, n) are astride the switching 
curve, and difference equations derived from the analytic 
boundary conditions are applied here. 

The boundary condition, co is continuous, may be applied 
simply. The condition / • n is continuous, may be written 

j Zl cos 9 — j Z2 sin 9 is continuous (21) 

where 0 is the angle between the z 2 axis and the switching curve 
at {m , n), and m is the generally non-integral value such that 

C(m'h,nk)= 0 

, Nodal pomts (m, n) will be eliminated in favour of points 
{m , n) on the switching curve whenever \rn — m\ < i or 
m - m = 1 Formulae (17) must then be replaced by the cor¬ 
responding formulae for unequal differencing intervals (For¬ 
sythe and Wasow 12 ) in the case of points (m, n) such that 
\ m — m\ < f or m — m = 

written™ s eqn (16) ^ COmponents of the flow vector can be 
Jz! Q ~ a ( Z l> Z 2 ) CO 

3*2= -b{z u z 2 )cD (22) 

and the condition (21) as 


The derivative dco/d/ l5 is replaced as follows: 
dco 1 \ 3 YTr _ TT7 1 1 

( 2 W m+l n ~ ^ffi + 2,uj j C ( Z 1 j Z2) = 0 + 

dco 1 f 3 1 1 

TT > ~f[ | ~2 Wm-l,n + ~2 5 Z 2)~0~ 

converting the continuity condition (24) to a difference equation 

~Y W n.-2,n + 2W m . Un 

- {3 + yh e mh [-a + +a~ +(b + -b~) Ian 0]}w m .„ 

+ 2~^/n + 2,n = 0 (26) 

for points (m, n) on the switching curve. 

Parameter y in (26) is at present gratuitous. 

Boundary Conditions on the Perimeter 

Another approximation will have to be made before the 
system of difference equations (20) and (26) can be solved. 
Smce the z space is unbounded, the system of equations is un¬ 
bounded unless m and n are confined within limits such as 


VS ~^ + aa> J cosS -bcosin9 is continuous (23) 

or m the (z/, z 2 ') space defined by (19), 


So 


\ e g z ^ +a ®y c °s0-h'o sin 6 is continuous (24) 
where z/ > 0; etc. 


It then becomes necessary to impose a boundary condition on 
the perimeter S defined as points (m, n) such that m — + r 
or n = ± s. ~ ’ 

One may attempt to set 

w m n = 0, ( m,n)eS (28) 

because of the prospect that for stable systems, 
iim co(z 1 ,z 2 )=0 

l*i|. \zi\-*oo 

However, (28) may not be a good approximation since in a 
physical sense, it implies that 5 is an absorbing boundary, 
tentatively, one could impose a reflecting boundary at S, but 
is too is unlike the analytic process where particles flow in 

° t J® Ctl0nS across s - A compromising condition will be 
adopted to simulate more accurately the analytic process. 

^+i,„ =ocW r , n 

and w -r-i,n =vW- r>n ^ 

W m,s + 1 =PW m , s 

an u ' 5 P0Sitive constants less than unity. The out- 

Portional W to y t °\ at the boundar y * = r is then pro- 
( x av ■ anc may zero ( x = 1) or maximum 

best Jw Af ° th ^ e f reme conditions considered above. The 
b,l“ d “ d § “ ° b,ai “ d W “■>»»«»• of Which 
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W 


The Normalizing Condition 

The normalizing condition (5) may be satisfied as follows. 
The condition may be rewritten for a finite boundary S as 

r s f r 

W(z 1 ,z 2 )dz 1 dz 2 = l (30) 

J — s J — r 

Set 

KW 00 = 1 (31) 

and solve for the relative values 

KW mn ; (m,n)*(0,0) 

and then determine the normalizing constant K from the 
integration (30). 


Empirical Determination of the Parameters oc, /?, y 

Setting KWw = 1, as in (31), necessitates the elimination of 
one equation from the system of eqns (20) and (26). If the matrix 
of the system is not degenerate* so that no equation is redun¬ 
dant, the elimination of the equation m, n = 0 gives the point 



Figure 2. Cross section W(C) displaying the possible source 
action of the point (( 0 , 0); - denotes the correct solution 



Figure 3. Cross section W(C) displaying the possible sink action 
of the point (i 0 , 0); - denotes the correct solution 

(0, 0) freedom to act as a source or sink of particles. If the values 
of a and /? are too low so that there is a net outflow of particles 
at the boundary S, the solution will have the character depicted 
in Figure 2, indicating that the point (0, 0) is acting as a source 
to supply the particles lost at S. If the values of oc and {} are too 
high, there is a net inflow of particles (by drift) at S, and the 
solution will have the character depicted in Figure 3 with the 
point (0, 0) acting as a sink. 

* Since the coefficients of eqns ( 26 ) and ( 29 ) (and ( 20 ) if c 4= 0 ) 
do not sum to zero, the matrix row sums cannot all be zero; because 
of the transformation ( 19 ) none of the column sums is zero; neither is 
there a sequence of numbers i = 1, 2, • • • such that if the i th 
row (column) is multiplied by X i for i — 1 , 2, ..., the column (row) 
sums are zero; therefore the matrix is not degenerate and there may 
be a sink or source at (0,0). 


Since the analytic solution o> (z l4 z 2 ) is known to be smooth 
along the switching curve, the parameters oc and 0 may be 
chosen so that the numerical solution W(z l9 z 2 ), C (z 1 z 2 ) = 0, 
is also smooth: so that the point (0, 0) does not act as a source 
or sink. 

Because of the nature of eqn (26), the points (m, n) may also 
act as sources or sinks. Small variations of the parameter y 
(y < 1) enable adjustment of W(z l9 z 2 ) until the necessary flow 
continuity is achieved without injection or absorption of par¬ 
ticles at the points (m, n ). The criterion of adjustment is again 
the smoothness of W(z x ,zf) for C (z l5 z 2 ) — 0: the point (0, 0) 
must not act as a source or sink. 

There is some indeterminacy in setting three parameters 
oc, /?, y from one criterion. However, it is helpful to know that 

lim oe,/? = 0 

r, s-* oo 

and 

lim y — 1 

h,k-+Q 

Also, the most effective parameter is y and oc and /3 may be used 
only for fine adjustments if required. It has been found in 
practice from examples for which analytic results are known 
that any set of parameters oc, /J, y which clearly satisfies the 
condition at (0, 0) produces a good approximation W(z lf z 2 ) in 
the sense of (15). 


Computation of the Solution 


The system of difference equations may be written as the 


linear equation 


AX = B 


(32) 


where A is a matrix, and X and B are vectors for the solution 
and right-hand sides. Eqn (32) has been solved on a digital 
computer by Gaussian elimination. A is a banded matrix and 
only elements inside the band need to be stored. Systems of up 
to 900 eqns have been solved for this problem, although it will 
be seen that fewer equations are required for useful analysis of 
second and third-order control systems. 


Examples of Numerical Solutions of the Fokker-Planck-Kolmo- 
gorov Equation 


A First-order System 

Equations of motion of the relay 
Figure 4 are . . 

e=-±-{e + y)-- 

• 1 ^ 

y=Y sgnC 


control system depicted in 

sgnC+— 

T (33) 


where C ( e,y ) is the switching function, and the input x has the 
spectrum iV 2 /( 1 +o> 2 1 2 ). 


' UuV 

Controller 

u 

JL 

PT 

y 












u = sgn (C (e,y)) 
e = x-y 
Figure 4 
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The Fokker-Planck-Kolmogorov equation and boundary 
condition for C (e,y) — 0 may be written 

2^"0? + {t ( e + + y sgn 4^ sgn c if + T w = 0 


N 2 dco {1 1 ) „ 

2t 2 de + ]Y^ e + y ^ + Y SgnC \ C0 + Y Sgn Cc ° iscontirmous ’ 

where g = de/dy 9 C(e, y) = 0. 

Eqn (34) has been solved numerically for the case N 2 = 2, 
T = r = 1 with the linear switching function, 


C(e, y) = e — Ky 

The error distribution obtained by numerical integration of the 
(e,y) distribution is shown in Figure 5. The mean squared error 
is given in Table 7, with some details of the numerical solution. 



Errors 



Figure 7. Linear versus parabolic switching 


A Second-order System 


As a second example, a numerical solution will be compared 
with the analytic solution obtained by Fuller 7 for the system 
depicted in Figure 8. The equation to be solved is 


N 2 c 2 co t dco 1 dco 

2 dP +Z de Y SgnC dI~° 


where z = y and e = x — y. 


Figure 5. Error distributions 

Table 1. Mean Squared Error and Parameters of the Numerical 
Solution for the Simple Integrator 


Control 

parameter 

K 


Cell sizes 


Grid 

dimensions 


Number 

of 

equations 


Mean 

squared 

error 

0-53514 

0*47416 

0*46258 

0*47940 

0-52198 


The optimum value of K is near TO, and the minimum mean 
squared error is 0*46. 

The accuracy of the computed ( e,y ) distribution has been 
tested by marginal integration of the (e + y,y) = (x,y) distri¬ 
bution, for the theoretically known x distribution. The numerical 
and theoretical results are in excellent agreement (Figure 6). 

The advantage of parabolic switching: C(e,y ) = <? — Ky \y\ 
has been investigated with the results shown in Figure 7. The 
minimum mean squared error is reduced by 2 per cent. 



u 

1 

1 — 


p 2 t 2 


u = sgn (e-ky) 
e = x-y 

Figure 8 

The analytic solution is given by 


^n 3 t 2 ^1 n 

and the error distribution by 


2 iv 2 r 2 


2 Ke 
exp—;—: 

n 2 t 


, -2 Ke 

+ exp——— 

N 2 T 2 


_ e ^ K 1 ' 1 

.N yic NT 2 


1+erf 
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g" 


Table 2. Numerical Solution for the Double Integrator K = 0*2 with 
Theoretical Values in Parentheses. The (e, y ) Distribution is Symmetrical 
with Respect to the Origin 


The (<?, y] 
Ay 

) distribut 

:ion: 





0-8158) 

(0-8194) 

i 

0-8568 

(0-8605) 

0-8138 

(0-8191) 

0-7605 

(0-7668) 

0-7004 

(0-7073) 

0-6332 

(0-6409) 

0-5597 

(0-5683) 

1 

0-8971 

(0-9044) 

i 

0-9280 

(0-9347) 

0-8698 

(0-8754) 

0-8136 

(0-8195) 

0-7491 

(0-7559) 

0-6769 

(0-6850) 

0-5981 

(0-6073) 

1 

0-9739 

(0-9761) 

i 

0-9496 

(0-9524) 

0-8985 

(0-9022) 

0-8394 

(0-8446) 

0-7722 

(0-7791) 

0-6974 

(0-7060) 

0-6159 

(0-6259) 

1 

0-9937 

(0-9951) 

0-9585 

(0-9605) 

0*9064 

(0-9099) 

0-8464 

(0-8517) 

0-7784 

(0-7857) 

0-7027 

(0-7120) 

0-6205 

(0-6312) 


1-0000 0-9546—0-9024—0-8424—0-7745—0-6991—0-6173-> 

(1-0000) (0-9567) (0-9063) (0-8484) (0-7826) (0-7092) (0-6288) (e) 



0-9413 

(0-9436) 

0-8895 

(0-8939) 

0-8300 

(0-8368) 

0-7632 

(0-7719) 

0-6890 

(0-6995) 

0-6083 

(0-6202) 


0-9110 

(0-9156) 

0-8610 

(0-8674) 

0-8041 

(0-8120) 

0-7397 

(0-7490) 

0-6677 

(0-6787) 

0-5892 

(0-6018) 


0-8585 

(0-8652) 

0-8127 

(0-8197) 

0-7592 

(0-7673) 

0-6980 

(0-7078) 

0-6295 

(0-6414) 

0-5550 

(0-5687) 


0-7785 

(0-7839) 

0-7353 

(0-7426) 

0-6857 

(0-6951) 

0-6296 

(0-6413) 

0-5673 

(0-5811) 

0-4998 

(0-5152) 

The normalizing constant: 





8-5734 

(8-6630) 






The error distribution: 





0-2661 

(0-2645) 

0-2595 

(0-2580) 

0-2453 

(0-2443) 

0-2291 

(0-2286) 

0-2107 

(0-2109) 

0-1903 

(0-1911) 

0-1679 

(0-1695) 

The mean squared error: 





1-5840 

(1-5909) 







The mean squared error is 


N 2 

2 



( 38 ) 


Tables 2, 3 and 4 present the numerical solution and theo¬ 
retical values for K = 0-2, 1-0, 1-8, together with numerical and 
theoretical values for the error distribution and mean squared 
error. 

The numerical solutions are obtained for a relatively small 
number of points, 59 or 68, to demonstrate the accuracy which 
can be achieved by a small system of equations. The estimate of 
mean squared error is in error by 0-4 to 1-3 per cent. 


A Third-order System 

A solution at 83 points for the system shown in Figure 9 is 
given in Table 5. 


X,— 

Controller 

u 

J 


y 



P 

1 3 



GII 

-- 





u = sgn(e-y-y) 
e = x-y 


Figure 9 


Table 3. Numerical Solution for the Double Integrator K = TO with 
Theoretical Values in Parentheses. The (e, >-) Distribution is Sym¬ 
metrical with Respect to Origin 


The (e, y 
Ay 

) distribu' 

tion: 





0-1518 

0-1867 

0-2397 

0-3242 

0*4673 

0*2843 

0-1560 

(0-1386) 

1 

(0*1729) 

(0-2266) 

(0-3153) 

(0-4719) 

(0-2883) 

(0-1579) 

1 

0-3250 

0-3992 

0-5117 

0-6909 

0-4646 

0-2872 

0-1592 

(0-3135) 

1 

(0-3912) 

(0-5126) 

(0-7132) 

(0-4765) 

(0-2911) 

(0-1595) 

1 

0-5343 

0-6575 

0-8462 

0*6178 

0*4191 

0-2599 

0-1446 

(0-5419) 

i 

(0-6761) 

(0-8861) 

(0-6368) 

(0-4254) 

(0-2599) 

(0-1424) 

1 

0-7600 

0-9445 

0-7301 

0-5292 

0-3570 

0-2215 

0-1238 

(0-7820) 

(0-9758) 

(0-7446) 

(0-5351) 

(0-3575) 

(0-2184) 

(0-1197) 


1-0000 0-7953—0-6053—0-4369—0-2957—0-1847—0-1039-^ 

(1-0000) (0-8014) (0-6115) (0-4395) (0-2936) (0-1794) (0*0983) (e) 



0-6129 

(0-6267) 

0-4726 

(0-4782) 

0-3451 

(0-3437) 

0-2357 

(0-2296) 

0-1482 

(0-1403) 

0-0838 

(0-0768) 

• 

0-4342 

(0-4342) 

0-3370 

(0-3313) 

0-2474 

(0-2381) 

0-1696 

(0-1591) 

0-1069 

(0-0972) 

0-0606 

(0-0532) 


0-2644 

(0-2512) 

0-2054 

(0-1917) 

0-1508 

(0-1378) 

0-1034 

(0-0920) 

0-0652 

(0-0562) 

0-0370 

(0-0308) 


0-1233 

(0-1110) 

0-0956 

(0-0847) 

0-0701 

(0-0609) 

0-0480 

(0-0407) 

0-0303 

(0-0249) 

0-0172 

(0-0136) 

The normalizing constant: 





3-4571 

(3-4460) 






The error distribution: 





0-3475 

(0-3492) 

0-3418 

(0-3439) 

0-3198 

(0-3220) 

0-2772 

(0-2791) 

0-2031 

(0-2029) 

0-1261 

(0-1239) 

0*0704 

(0-0679) 

The mean squared error: 





1*1112 

(1-0971) 







Statistical Linearization 


By the method of Booton 14 for the statistical parameters of 
non-linear systems, the mean squared error for the second-order 
system is given by 


N 2 / n N 2 T a 

t!,4T + ‘ 
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Table 4. numerical Solution for the Double Integrator K = T8 w ith Theoretical Values in Parentheses 
__ The Distribution is Symmetrical with Respect to the Origin 


The ( e , y) distribution: 



A • 

j y 




0-0236 

(0-0266) 

| 

0-0349 

(0-0379) 

0-0554 

(0-0580) 

0-0954 

(0-0964) 

0-1135 

(0-0966) 

| 

0-1616 

(0-1377) 

0-2440 

(0-2106) 

0-3944 

(0-3503) 

0-2898 

(0-2593) 

| 

0-4022 

(0-3698) 

0-5898 

(0-5656) 

0-8740 

(0-8799) 

0-5547 

(0-5547) 

0-7641 

(0-7911) 

0-9258 

(0-9733) 

0-4678 

(0-4706) 


0-1817 

(0-1773) 

0*4854 

(0*4851) 

0-2654 

(0-2674) 

0-0846 

(0-0940) 

0-7420 

(0-7106) 

0-3919 

(0-3780) 

0-1638 

(0-1579) 

0-0534 

(0-0555) 

0-4506 

(0-4475) 

0-2228 

(0-2158) 

0-0960 

(0-0901) 

0-0324 

(0-0317) 

0-2635 

(0-2559) 

0-1325 

(0-1234) 

0-0578 

(0-0515) 

0-0198 

(0-0181) 


1-0000-0-7047-0-4677-0-2886- 

(1-0000) (0-7012) (0-4584) (0-2756) 


-0-1627- 

(0-1499) 


-0-0819- 

(0-0723) 


-0-0356- 

(0-0302) 


- 0 - 0122 - 

(0-0106) 


(e) 



0-4014 

(0-3890) 

0-2717 

(0-2543) 

0-1695 

(0-1529) 

0-0957 

(0-0831) 

0-0478 

(0-0401) 

0-0205 

(0-0167) 

0-0070 

(0-0059) 

* 

0-2077 

(0-1818) 

0-1386 

(0-1189) 

0-0848 

(0-0715) 

0-0468 

(0-0389) 

0-0228 

(0-0187) 

0-0095 

(0-0078) 

0-0032 

(0-0028) 


0-0793 

(0-0677) 

0-0514 

(0-0443) 

0-0305 

(0-0266) 

0-0162 

(0-0145) 

0-0076 

(0-0070) 

0-0031 

(0-0029) 

0-0010 

(0-0010) 


0-0159 

(0-0186) 

0-0099 

(0-0122) 

0-0057 

(0-0073) 

0-0029 

(0-0040) 

0-0013 
(0-0019) j 

0-0005 

(0-0008) 

0-0002 

(0-0003) 

The normalizing constant: 






1-6140 

(1-5497) 






j 

The error distribution: 







0-3418 

(0-3431) 

0-3375 

(0-3406) 

0-3344 

(0-3371) 

0-3018 

(0-2999) 

0-2633 

(0-2630) 

0-1716 

(0-1711) 

0-0764 

(0-0756) 

0-0354 

(0-0358) 

The mean squared error: 







1-0237 

(1-0198) 

. I 


- 





Tabk 5. Numerical Solution for a Third-Order System with Symmetry About, the Origin 


0-031 

0-045 

0-060 

0*078 

0*097 

fy 

0-118 

0*143 

0-077 

0-225 

0*212 

0-171 

0-051 

0-075 

0-102 

0*113 

0*168 

| 

0*207 

i 

0*255 

0-324 

0-320 

0*288 

0-226 

. 0-078- 

-0-1 IS¬ 

—0-159- 

—0*212- 

—0*273- 

1 

-0-344- 

| 

—0*435- 

—0-444- 

—0-435- 

—0*396- 

—0-316--> 

0-115 

O-172 

0-241 

0*325 

0*422 

0.531 

0*550 

0-553 

0-532 

0*477 

e 

0-378 

0-158 

0-230 

0-313 

0*402 

0*490 j 

j 

0.486 

0*482 

0-473 

0-448 

0*400 

0-316 

0-001 

0-011 

0-031 

0*060 

0*096 

y= 

A 

0*1 

1 

A7 

k ** 

y 

37 

0*189 

0-263 

0-275 

0*255 

0-201 

0-026 

0-053 

0-097 

0*158 

0*236 

0-330 

1 ■ 

0*454 

0-466 

0.394 

0.293 

0.196 


0-104—0*181-0*300-0*470—0*701_1*000— * 
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and the optimum value of k as 

-JV 2 T 4 

The correct values are respectively 

n 2 (n 2 t a 


+ k 


2 V k 2 
(2 N 2 T A f 

For the first-order system, the method of statistical lineariza¬ 
tion gives 

<x e 2 in =0-398; 

by numerical solution the following is obtained: 

o-4m=0-46; 


and 


fe„ pt = 0-6 


AW* 1-0 


Conclusions 

The numerical method described in this report is a means by 
which the analysis of simple non-linear control systems with 
Gaussian inputs may proceed. Solutions of useful accuracy can 
be obtained on present sizes of digital computing machines for 
systems of two or three coordinates, and high accuracy is pos¬ 
sible for systems of two coordinates. 

Solutions for such simple systems can provide unknown in¬ 
formation such as the nature of the distribution of signals in non¬ 
linear control systems, and the gain to be expected from non¬ 
linear control, the validity of linear approximations, and the 
accuracy with which a given set of transition probabilities de¬ 
scribes a continuous random process. 

The writer is indebted to Dr. A. T. Fuller for much helpful 
supervision. 
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DISCUSSION 


P. C. Parks, Department of Aeronautics, The University, Southampton 


As Dr. Merklinger remarks, there are very few known solutions of the 
Fokker-Planck-Kolmogorov equation. Those solutions which are 
known, for example that for linear systems due to Wang and Uhlen¬ 
beck (Reference 4 of his paper), the system of Barrett’s Figure 3 
(Reference 7 in the paper) and the result of Fuller [eqn (36)], take the 
form 


co = Dexp 



where N 2 is the power spectral density and V is a Liapunov function 
which may be used to prove that the undisturbed system has a stable 
origin, D being a normalizing constant. 

One new explicit result found by this approach is the phase space 
steady-state probability distribution for the nth order linear equation 


cfbc 
d f 


+ Qi 


d" Y x 

d r 1 


+ <%2 


d"~ 2 x 

dt"~ 2 


+ ... 


+a„x = £ 


where x r — (x, x, x,... x^ n ^), the phase space state vector, 

D=VAAT /(^) n/2 

and the elements Pij of P are defined as follows: 


n-i 


p 0 =L(- 

i- j-k+i i+jeven 

k~ 0 

0'^j) 

c; 

11 

0 

i +j odd 

p..=p.. 

r ji 1 ij 

0<j) 


a Q = 1, or = 0 if r > n. A n and A„- x are the 72th and (n — l)th Hur- 
witz determinants of the nth order linear equation. 

The matrix P was discovered by Ralston 2 and the quadratic form 
x'Px provides also a proof, in two lines, of the Hurwitz stability 
criterion using Liapunov’s second method 1 . 
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where f is Gaussian white noise. The result 1 is 


o = Dqx p 
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V. S. Pugachev, Inst, of Automatics and Telemechanics of the Ac. Sci. 
of U.S.S.R., Moscow , 1-53 , Kalanchevskaya, 15a, U.S.S.R. 


The paper presented by Dr. Merklinger is an interesting extension of 
studies of non-linear systems with the aid of the theory of Markov 
random processes. The only remark I should like to make is that it 
would, perhaps, be preferable to obtain a numerical solution of the 
integro-differential equation given in my discussion of Dr. J. F. Bar¬ 
rett’s paper at the 1st I.F.A.C. Congress, Basle, for the process charac¬ 
teristic function g (/; A): 


S g(A;t) 

0 / 


(2 n)" 


dx 


(k — fi) x 




rather than that of Fokker-Planck-Kolmogorov equation. Since the 
right member of the above equation is simply the expectation of the 
random variable e (A; /, X) for fixed t,X=X (t) being the Markov 


process understudy, it is possible to evaluate the integral in the above 
equation by means of the Monte Carlo method at each stage of the 
numerical solution. The volume of necessary calculations will then 
increase less rapidly with the order of the original set of differential 
equations, than in the case of the numerical solution of the Fokker- 
Planck-Kolmogorov equation, and this may present some advantages. 


K. J. Merklinger, in reply 

Mr. Parks’ solution for linear systems is more explicit than any pub¬ 
lished hitherto. 

Professor Pugachev’s suggestion to apply numerical methods to 
the integral equation is quite interesting, and could prove to be more 
economical. However, since an algorithm for the integral equation 
would be without physical significance, there might be some difficulty 
with convergence, boundary conditions, etc. 
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Volterra Series Representation of Time-varying 
Non-linear Systems* 

R.H. FLAKE 


Summary 

In this paper the concept of a functional Taylor series is applied 
directly to the functional relation between the solution and forcing 
function of a large class of differential equations, yielding the Volterra 
series. A simple, convenient method is presented for calculating the 
Volterra kernels in the Volterra series representation of responses 
with zero and non-zero initial conditions, for a broad class of time- 
varying non-linear physical systems. 

Sommaire 

Dans ce rapport, on utilise le concept d’une s6rie fonctionnelle de Taylor 
pour representer la relation fonctionnelle entre la solution et la sollici- 
tation d’une grande classe d’equations differentielles, donnant des series 
de Volterra. On montre une methode simple pour calculer les noyaux 
de Volterra dans la representation, par sdrie de Volterra, des reponses 
avec des conditions initiales nulles et non nulles, pour une large classe de 
systemes physiques non-lineaires a coefficients variables avec le temps. 

Zusammenfassung 

In diesem Beitrag wird der Begriff der Funktional-(Taylor-)Reihe un- 
mittelbar auf die Funktionalbeziehung zwischen der Losung und der 
Anregungsfunktion einer groBen Klasse von Differentialgleichungen 
angewendet; dabei ergeben sich Volterrasche Reihen. 

Die Arbeit enthalt ein einfaches, bequemes Verfahren zur Berech- 
nung des Volterraschen Kernes in den durch Volterrasche Reihen dar- 
gestellten Antworten. Dies gilt fUr eine grofie Klasse von zeitverander- 
lichen nichtlinearen physikalischen Systemen mit sowohl verschwin- 
denden als auch nicht verschwindehden Anfangsbedingungen. 


Introduction 

The Volterra series represents, for non-linear systems, the gen¬ 
eralization of the concept of a transfer function, which is of 
primary importance in the analysis and design of linear systems. 
The Volterra series represents an explicit input-output relation 
for non-linear systems, and consists of an infinite series composed 
of terms of the form of convolution integrals. The first teim is 
the convolution integral of the first order kernel and the forcing 
function of the differential equation describing the system. The 
first-order kernel is the impulse response of the linear portion 
of the non-linear differential equation describing the system. 
The «th order term is an 71 -fold convolution integral containing 
the /ith order kernel multiplied by an nth order product of the 

forcing function. . 

Historically, this type of series first appeared around 1910 m 
the studies of Volterra 1 and some of his contemporaries on 
functional equations. More recently it has been studied by 
Wiener 2 , Brilliant 3 , George 4 , McFee 5 , Blackman 6 , and others. 
Zadeh 7 ’ 8 considers a generalization of the Volterra series. 


Volterra, Brilliant, and Blackman present proofs showing 
that, under certain conditions, a functional y(t) = T'frlOl can 
be approximated to any desired degree of accuracy by a finite 
series of the form 


T0) = j h 1 (x)x(t-x)dx 

+ P f h 2 (x 1 ,x 2 )x(t-x 1 )x(.t-x 2 )dx 1 dx 2 + ... 


+ 


0 J 


h n (T 1 ,...,T„)x(t-Tjx(t-T 2 ) 


... x(t—x„)dx 1 ...dx„ + ... 


( 1 ) 


Such a functional is called continuous 1 ’ 3 - 6 . It is easy to show 
that the functional relation between the solution and the forcing 
function of a non-linear differential equation with constant co¬ 
efficients, satisfying the Lipschitz condition, is continuous. If 
T [jc( 0] can be represented exactly by a converging infinite series 
of the form of eqn (1), it is called analytic 1 . Volterra shows that 
eqn (1) can then be interpreted as a functional generalization ot 
the Taylor series expansion for the analytic functional. Con¬ 
ditions for convergence of the Volterra series are considered by 
Volterra and Brilliant. 

One of the main problems in the application of the theory is 
the explicit determination of the kernels appearing in the Volterra 
series. Both analytical and experimental procedures for deter¬ 
mining these kernels for time-invariant non-linear systems have 
been investigated 4 ’ 5 - 9 and the last two sections of this paper 
show that the problem of calculating the Volterra kernels as¬ 
sociated with time-varying non-linear systems may be ap¬ 
proached in basically the same manner as the method for time- 
invariant systems described by Flake 9 . As in the previous paper , 
a Volterra series representing the response of a non-linear system 
with zero initial conditions is first considered here, and then 
modified to represent a response with arbitrary initial con¬ 
ditions. In the first part of this paper, Volterra’s treatment of 
the functional Taylor series for an abstract analytic functional 
is applied directly to the functional relation between the solution 
of a forced non-linear differential equation and the forcing 
function. 


The Functional Taylor Series 

Define the functional, = T[y], by the differential equa- 


tion 1 ', 




( 2 ) 


* This work contains part of the results of a dissertation submitted 
by R H Flake in partial fulfilment of the requirements for the degree 
of Doctor of Science in Engineering at Washington University. 


t The results in the following are easily generalized to a system of 
differential equations which can be written in the form of eqn (2) by 
using matrix notation. 
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where y(t ) is the set of continuous functions with continuous 
first derivatives defined over the interval 0 < t < a, with 
,v( 0 ) = 0 . fit, y ) is a continuous function of t and an entire 
function of y for each value of t, with fit, 0) = 0. The solution, 
of eqn (2) is given formally by T~ l [(pit)] = F[<f>(t)]. 

Now suppose the forcing function, (pit), is changed by an 
amount exit), where x(t) is a continuous function and e is a 
parameter. Then the solution becomes 

T £ ( t) = F[tj> (t) + ex(f)] 

If t, (pit), and xit) are assumed fixed, and t is sufficiently small, 
then y s may be considered an ordinary function of e 1 . Assuming 
y E has continuous derivatives up to nth order with respect to £ 
on the interval 0 < e < b, then, by Taylor’s theorem (actually, 
y £ (f) is an analytic function of £ for t sufficiently small 10 ). 


eqn ( 2 ), both satisfying zero initial conditions. Let I y 
F [(p it) + £ xit)] - F [(p it)]. Then 11 

j: ( A y ) =f(t, y+Ay)-fit, y) + ex it) 


fo(Ay) = Ay j ^ f y it, y + rjA y )djy + sx (/) 
where/„ is the derivative of f(t,y) with respect to y. 

l 

Now let Ait) = //„ it, y + rj Ay) chj, and consider the 
o 

solution /(t,t') of 

d l 


y e =y c !a=o+|~ • e 2 / 2 ! +... 

8e £=0 0 £ - £ ^ 0 ' 

, g-VJ o'- 1 9”y £ rf* , 

38"- 1 L=o(n-l)! 0 e » £=fl n! (3) 

where 0 < 6 < e < b. The terms in eqn (3) may be interpreted 
as o lows. y & (t) is the solution of the differential equation 


av_. 

( 0 ) + $ (0 + sx (0 


satisfying the initial conditions y t ( 0 ) = 0? The first term on the 
right side of eqn (3) is the solution of eqn (4) satisfying zero initial 
conditions when e = 0 , and is therefore a solution of eqn ( 2 ) 
The second term on the ri^ht side of eqn ( 3 ) in a multiplied by 
the quantity J 

9 jd =ij m — ( ? )+ ex CO]—FT<Kt)l 


where /(?',/') = 1 . Then, multiplying both sides of the differen¬ 
tial equation above for Ay by / yields 

, d d l , 

l-Ay=-Ay- + sl X 

■» , M 

or t f /v 

l ( t , t')Ay(t) = e / (r 5 1') x (t) di 

o J o 

t* 

Finally, Ayit’) = e//(r,/') x(t) dr, since ly( 0 ) 0 , and 

l(t’,t') = 1 . ° 

Now 

SF \jp, x] =lim —— 

£~>0 & 
f*t 

=lim l (t, f) x (t) dt 

e->0 J 0 


H-O'Z - r • © 

which is called the functional differential of F with respect to 
x( ), evaluated at (Pit) and denoted by dFty.x]. This quantity 
will presently be explicitly calculated. 

The higher order functional differentials appearing in eqn ( 3 ) 

are obvious generalizations of (5). Thus, eqn ( 3 ) is a functional 

generalization of the Taylor series for a solution of eqn ( 4 ), ex- 

l 6 , d a . b °, Ut a soIution of ^ (2)- Note that, if a solution of 

f“ ( atlS y “ g ceitam conditions is known, thenevalua- 

ime inT,, ^ W H v yid r the S0lUti0n 0f eqn (4) satisf ying the 
same rnitial conditions. In particular, if 6(f) = 0 then the 

so ution of eqn ( 2 ) satisfying zero initial conditions is y(t) = 0 

and therefore an evaluation of eqn (3) would yield the solution of 

dy 

'fa=f( t >y)+ex(t) 

satisfying the initial condition, y( 0 ) = 0 . 

Calculation of the Functional Differential 

The functional differentials appearing in eon m „ct „r 

“j" - cZSLXi o' 

e x(tA k ! he &St dlfferentla l given by eqn (5). F [d>it) + 

,x(m “ a sol "'“"° f w. rim «„ + f 


Let /„ be defined as the solution of 

d/ 0 

dt A o(t)l 0 ( 6 ) 

with /„ it', f) = 1 , where A 0 0) = Urnff, it, y + t , Ay) d/y. But 

j s ->0 0 

limdy = 0 . Thus, or A 0 (t) =/„ (t,y(r)), 

ditlSTIndly, 8011111011 ° f Cqn (2) SatiSfying Zer ° initia! Con - 

rt 

[<p f x] = l 0 (t, t ) x (t) dr (7) 

4 / 0 

Before evaluating eqn (7) for a particular example, the second 
differential will be calculated. 

Consider 

5 2 F r 6. xl =li m jfl - ^ 14>, *1 

£~* 0 fi 

Now 8 F[<p, x ] is given by eqn (7); and similarly, 

<5F O + ex, x] f (t, f) x ( x ) dT 
where 4 (r, t') is the solution of 
’ d/i 

dt T e ) h 
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satisfying the initial conditions l ± t') = 1. Thus 


s 2 f [>,*]■ 


’ li[n 

0 £->0 \ £ 


( 8 ) 


Let zl / = 4 — l 0 . Then 
d 


or 


d t 


J (Al)=-B(t)Al-C(i)l 0 


df 


= B(f)m, with m{t, f")| t=( „= 1 


Then, multiplying eqn (10) by m it, t") yields 

m $: (A /) = - A l ^ - C (t) m (t, t ") Z 0 (t, t‘) 


d t 

Integrating yields 
m (t, t") A l(t, t') 


t" 


or 


A l (t", 0= 


dt 

m (t, t") C (t) / 0 (t, t') dt 

t" 

m (t, t") C (x) l 0 (t, t') d t 


Thus, 


e _> o 


—=J dt m 0 (t, t ) l 0 (t, O/yy Oh T to) [> (t)] 


and, from eqn (8) 
Finally, using eqn (7) 


Eqn (11) 


Eqn (11a) f 


J t (d l) = -fy (t, ye) ll +fy (t> y) l 0 


iAl ) = -f y it, y s )Al — [fyit, y + Ay ) -f y it, y)] l 0 (9) 


where / 0 0, t r ) is defined by eqn (6), and m Q ( t , *") is the solution of 


d m 0 
dt 


=fy it, y (0) m 0 > m O ( t, t") I t = t» = 1 (12) 


Now, f y it,y + Ay) — f y (t, y) = A y ( f vy 0, y + r\ A y) d rj. 

0 

Then eqn (9) becomes 


The higher order differentials of Fmay be calculated in a similar 
manner. 

Eqn (11a) may be written in the form 


( 10 ) 


5 2 F [4>,x] = 2l 


h 2 (t 9 % 9 rj)x(T)x(ri)dxdfj (13) 


where B (t) = f y (t, y s (/)), and C (t) = A y ( f yy ( t,y + A y) d rj. 

o 

Consider the solution of 
dm 


if h 2 (/, r, rj) is defined (u(t) is the unit step function, which is 
equal to one for t > 0, and zero for t < 0) as 


Eqn (13a) =(= 


or 


Eqns (14 , 14a) § 


Volterra 1 writes the /ith order differential in the form 
S”F[f,x] 


— n ! 


Kit, t l5 ..t„) x (t t )... x (t„) dt!... dt„ 


Thus, eqn (3) may be written as 


T s (0 = e h 1 (t,z)x(x) dz 


+ £ 2 h 2 it, z 1 ,r z )xiT 1 )x (t 2 ) dti dt 2 +... 


(15) 


+ -^<5"F[<M0 + 0x(t)] 


Eqn (11) 


f Eqn (11a) 


=4= Eqn (13a) 


§ Eqns (14 , 14a) 


<5 2 F [0, x] = dzx (t) d dm 0 (0, z) l 0 (9, t) f yy ( 9, y ( 9 )) §F [c/> (0)] 


5 2 F [<M] = 




dtx (t) ■ d0m o (0, t) l 0 (0, f) / vy (0, y (0)) 


dj7/ o (»7,0)x(/7) 


^(4 L'7) = 


or h 2 (t, t, rj) = 


2! 


1 

2 ! 

ir* 

2 ! 


m 0 (0, t) Z 0 (0,0 / yy (0, y (0)) uiO-t]) l 0 in, 0) d0 
m 0 (0, t) Z 0 (0,1) /yy (0, y m lo (v,0)d 0 if t > J? 

m 0 ie,x)l 0 ie,t)fyyid,yid))l Q in,d)dd if toy 


( 11 ) 

(Ha) 

(13a) 

(14) 

(14a) 


93 








R. H. FLAKE 


where 0 < 6 < e. If the number of terms is allowed to become 
arbitrarily large, then the infinite series is called the Volterra 
series. Note that the last term in eqn (15) may be used to estimate 
the error in the approximate evaluation of y e for a series com¬ 
posed of a finite number of terms. An upper boundary on the 
remainder term may be calculated, provided an upper boundary 
on y £ (t) is known. 

Example Consider the differential equation 

^f+y e + Ky 2 e =sx(t) (16) 

The first two terms in the functional expansion of y e about zero 
forcing function (<f> (t) = 0), satisfying zero initial conditions, 
are now calculated. l 0 (t , f) is the solution of eqn (6), which in 
this example becomes 

dZ 

-^=/ 0 , with Z 0 (/', t') = l 


dx 1 


dr 2 e (t T2) + 

dr,e Tl) 

J o 


1 o 

'ti ■ J 


or 

j £ (0 = e(l-e- , )-^8 2 (l-2te-‘-e- 2r )+... (19) 

h x (t, r), h 2 (t, Tj, t 2 ) and the higher order kernels in the 
Volterra series are called Volterra kernels. A more direct and 
convenient method for calculating the Volterra kernels is given 
in the next section. 


Determining the Volterra Kernels 

A simple, convenient technique is introduced in this section 
for calculating the Volterra kernels associated with physical 
systems which can be described by ordinary differential equations 
of the form 

L [y (0] + F [t, y (t), y' (t ),- 1 > (f)] = x (t) (20) 


since A 0 (t) -1. Thus, /„ (t, t") = e*-*' for t < t' , and eqn (7) 

becomes 

<5F[0,x]=f e z ~ c x (t) dr (17) 

or h 1 ( t , t) = t T 1 . m 0 (t, t") is the solution of eqn (12), thus 
dm 0 

—^-= - m 0 , with m 0 (f, t") = 1 
Therefore, m 0 = an( j eqn (14) becomes 

h 2 0»Tl,T 2 ) 

=J\ | m o (8, ti) lo (0, t) f n ( 9 , y (9)) l 0 (t 2 , e ) d 6 
=~(~2K)^ e -< fl - T ‘) e (9 - f >e< T2 - e )d0 

= K[e^'-^e-(‘-^- e -('-^] for >t 2 (18) 
Similarly, eqn (14a) yields 

h 2(t,T : i,T 2 ) 

==J? [e-h-n) e -(«-x 2 )_e-d-u)- j forTi<T2 (lga) 

Thus, if x(t) is a unit step function, then the first two terms in 
eqn (3) become 


y e (t)=s e T ‘dT+s 2 i<: 

'0 J oj 


e -(‘-n)p-(f-T 2 ) 


e dtidta 


where £ is a linear differential operator with variable coefficients, 
and F is a polynomial in y (t), y' (/),..., y <»-i> (/), beginning 
with terms of order no lower than the second. (The treatment 
could be applied to the more general vector form of eqn (2). 
However, equations of the form of eqn (20) are considered here 
for the sake of simplifying the notation.) The coefficients of the 
polynomial F are, in general, continuous functions of t. n is the 
order of the highest derivative of y (t) appearing in L [y (/)]. 
x ( t) is a continuous function. 

The basis of the technique is the assumption that the partic¬ 
ular solution of this class of forced non-linear differential 
equations can be represented by a converging infinite series of 
the form 

\~ Eqn (20a) | * 


where the kernels h n (t, r lt ..., r.„) are continuous functions, 
symmetric in r t -. The particular solution for an nth order differ¬ 
ential equation represented by eqn (20a) is, of course, the one 
for which the solution and its first n — 1 derivatives are all zero 
at t = 0. A modified form of eqn (20a) is later used to 
represent solution with non-zero initial conditions. 

The proposed procedure for calculating the kernels in a 
Volterra series solution of eqn (20) is to substitute eqn (20a) into 
eqn (20) and equate the coefficients of the different functionals 
of x (/). Since x (t) is an arbitrary forcing function, then the 
coefficients of each functional of x (f) must vanish if eqn (20a) is 
to be a solution of the differential equation. The kernels in the 
Volterra series solution are uniquely determined by the resulting 
set of equations. Detailed calculation of the kernels for a simple 
example will clarify the procedure. 

Let the system of interest be described by the first order 
differential equation 

A ( t ^+ B ( t )y+Ky 2 =x(t) ( 21 ) 


* Eqn (20a): 

y(t) 


| o / Jl (l,t)x(T)d t + j|^ 2 (t jTljT2 ) x(T i) x(T 2 )dfidf2+ ...,t„)^(T 1 )... 


x ( T n)dtj... dx„ + ... (20a) 
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Substitution of eqn (20a) into eqn (21) and equating the coef¬ 
ficients of the different functionals of x (0 yields 


Eqn (22-21) 


These equations may be solved sequentially for the kernels. 
Eqns (22) and (23) yield 




(t,x) = 


1 


= 0 


n ap) 

JtA(0) 


dO 


for t>x 
for t< x 


(28) 


h 2 (t, r l9 t 2 ) is determined from eqn (25) and the boundary 
conditions determined from eqn (24). Note that eqn (25) is a 
non-homogeneous form of eqn (23). Thus h x (t, t) can be used 
as a one-sided Green’s function 12 ’ 13 for eqn (25). Therefore 


h 2 (t,x x , t 2 ) — 


d6G(t,9)h 1 (d i x 1 )h 1 (e,x 2 ) 

0 


(29) 


where G ( t , r) = h x ( t , r) for t > r. Eqn (29) is a solution of 
eqn (25) in terms of the previously determined first order kernel, 
and it satisfies the boundary conditions given by eqn (24). The 
nth order kernel h n (t, r x , ..., x n ) is determined in terms of the 
kernels of order less than n from eqns (26) and (27). That is 




-T 


d 0G(f,0) 


n- 1 


X hj (0, 'C 1 ? • 1j) h n -j (09 15 •* *5 v) 

J=1 


for n = 2, 3,... 


(30) 


Thus the higher order kernels may be obtained from repeated 
use of eqn (30). 


In general, the Volterra kernels associated with a differential 
equation of the form of eqn (20) are determined by the equations 
resulting from equating the coefficients of the different func¬ 
tionals of x (0 that appear when eqn (20a) is substituted into the 
differential equation. The first order kernel is recognized to be 
the impulse response of the linear portion of the differential 
equation. The higher order kernels are solutions of non- 
homogeneous linear partial differential equations, satisfying 
associated boundary conditions. The solution of the equation 
determining the nth order kernel may be expressed in integral 
form as the product of the first order kernel, used as a one-sided 
Green’s function, and kernels of order less than n . Thus, if the 
first order kernel is known, then the determination of the 
analytical form of the higher order kernels depends upon being 
able successively to evaluate integrals containing previously 
determined lower order kernels. (If the linear portion of the 
differential equation has constant coefficients, then the first order 
kernel may be obtained by routine calculation.) Note, however, 
that a closed form, or, indeed, any exact solution for the first 
order kernel, may not be obtainable, since in general it is the 
impulse response of a linear differential equation with variable 
coefficients. Nevertheless, it is often possible to obtain a suitable 
approximate expression for the first order kernel, which could 
then be used to calculate approximate higher order kernels. The 
resulting approximate Volterra series solution may be sufficiently 
accurate for the purposes of the analysis. 


Non-zero Initial Conditions 

The general solution of a differential equation of the form of 
eqn (20) may be represented by a Volterra series modified by the 
introduction of a pseudo-forcing function. 

Consider first the generalization of eqn (20a) to represent a 
solution of eqn (20) which has some arbitrary non-zero value 


* Eqns (22-27): 

Functional Coefficient 


x(t) 


A(t)hi(t, f) = l 

(22) 

*t 

dxx (t) 

J 0 


(23) 

*wj 

dxx (t) 

1 0 

A (0 [fc 2 (t, r,t) + h 2 (t,t, t)] = 0 

(24) 

J JdT 1 dT 2 x(T 1 )x(T 2 ) 

A(t)^jh 2 (t,r 1 ,r 2 ) + B(t)h 2 (t,T: 1 ,T 2 ) + Kh 1 (t,r 1 )h l (t,r 2 )=0 

(25) 

x(0 

ft r n 

• •• n dT ;X<>;) 

0 J i — 1 

i*j 

n 

A(t) ^ h n (f, x x , .. Xj- 1 , t,Xj + x , ..., x n ) 0 
;=1 

(26) 

r,~ 

% n 

n dr i x < T i) 

J i= 1 

n ~ 1 

A(t)^-h n (t,x i ,...,x n )+B(t)h n (t,x 1 ,...,x n ) + K £ hj(x l ,...,xj)h n „j(x j+i ,.. 
O t j-1 

fT 

0 

il 

'Ga 
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initially, and whose first n — 1 derivatives are initially zero. 
This may easily be accomplished by introducing an auxiliary 
variable which is the difference between the desired solution and 
its initial value. This new variable will satisfy a differential 
equation of the form of eqn (20), if the terms in the new differen¬ 
tial equation which do not depend on y (/) are included in the 
forcing function. The new variable may then be represented by 
a series of the form of eqn (20a). Finally, the desired solution is 
the sum of the series for the auxiliary variable and an arbitrary 
constant. The generalization of the Volterra series to represent 
a solution for a second or higher order differential equation 
satisfying non-zero initial conditions on its derivatives, is perhaps 
not as obvious. A particular example will first be presented, and 
then the procedure for calculating the modified Volterra series 
representing the general solution of an «th order differential 
equation of the form of eqn (20) will be stated. 

Consider the second order non-linear time-varying differen- 
tial equation 

d 2 y dy 0 

i? +t di +y+ y =x (o (3D 

A procedure for calculating a modified Volterra series form of 
solution satisfying the initial conditions 

T(0) = 0 

dy 

dF 1=0 = * (32) 

is now shown. 

Define /( t ) by the relation 


The solution of eqn (37) satisfying the initial conditions given 
by eqns (35) and (36) is 


h 1 (t,x)=e ‘ 2/2 


2 dr + e 12,2 e l2/:2 


The second and higher order kernels may be expressed in 
terms of a one-sided Green’s function and lower order kernels 
The one-sided Green’s function is determined from the impulse 
response of the linear portion of eqn (32). From eqn (38) (39) 
and (40) it follows that 

h 2 (t, T U t 2 ) =-J o dOG (t , 6) h x (9, Tl ) h t (6, r 2 ) (42) 
where G (t , t) is given by 


df +/=x(r) 

and assume a solution for eqn (31) of the form 


y(t)=\ h 1 (t,z)f( T )dT 


+JoJ h 2( t ^i^2)f(ri)f(T 2 )dT 1 dx 2 + ... (34) 

Then substitution of eqns (33) and (34) into eqn (31) and equating 
the different functionals of/(r) as before yields 

h t 0,0 = 1 (35) 

0 

gjMf,T)| t = ( =l -t (36) 

6 

qP — + t -Qjh 1 (t,r) + h 1 (t,x)=0 (37) 

h i(t,T:,t) + h 2 (t,t,T:) = 0 (38) 

0 0 

dth 2 (t, X x, t) i tl . ( +— h 2 (f, T, Tl ) [ ti = ( =0 (39) 

0 2 0 

gP«aftTl»T a ) + ^A 2 (#, Tl ,T 2 ) 

+ h 2 ( t , t 1 , t 2 ) + h x ( t, r x ) h 1 (t, t 2 ) = 0 


G(f,x)=e 




z rl/1 dr, forOr 


and h x ( t, x ) is the previously determined first order kernel. The 
igher order kernels may be determined in the same manner. 
It is easily seen by inspection that the modified Volterra series 

® satisfy eqn (31) and the desired initial conditions 
if/(0)=y 1 . 

The proposed procedure for obtaining a modified Volterra 
series representing the general solution of eqn (20) is first to 
introduce an auxiliary variable, u (f), which has zero initial value 
and differs from the desired solution by a constant, u ( t ) will then 
satisfy a differential equation of the form of eqn (20), if all terms 
which are independent of u(t) are included in the forcing 
function. A solution of the auxiliary differential equation can 
be expressed as 


w 0) = Jo^i(^^)/(T) d T 


+ JoJ * 2 ^ T i* T 2 )/( T i)/('r 2 )dTidT 2 + ... (44) 

where/(/) is the general solution of 
d (n “ 1) df 

di (»-1) ^(0 + "-+^(0+/ (0= z (0 (45) 

and z {t) is the forcing function of the auxiliary differential 
equation. If the impulse response of the linear portion of the 
auxiliary differential equation can be found, then it can be used 
as a one-sided Green’s function to calculate the higher order 
kernels in eqn (44), by the procedure previously described 
Finally, the general solution of eqn (20) will be 


y(t) — C+j ^lC^'O/COdr 


Jo J ^ Tl 3 T2 > /( t 2) dr 2 + ... 

where C is an arbitrary constant. 

The author deeply appreciates the numerous helpful discussions 
with Professor John Zaborszky during the course of this study. 
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DISCUSSION 


J. F. Barrett, Mechanical Engineering Department , Birmingham 
University, Birmingham, England 


I would like to point out an alternative method of calculating the 
Volterra series expansion for certain non-linear differential equations. 
This method is applicable to the examples considered in Dr. Flake’s 
paper. The method is based on the conversion of the differential 
equation into a non-linear integral equation followed by series solution. 
When applicable it gives the Volterra series with little calculation. It 
has the advantage that it permits proof of the Volterra series, which is 
a very important point since convergence is directly related to stability. 
To illustrate the method consider the first example of Dr. Flake’s paper 


Then 


j} + y4- Ky 2 = sx (t), t > 0 
y + y — sx(t)-Ky 2 


(left-hand side linear) 

y(t)=y 0 e~ t + e 

't 




x (t) dr 


-K 




y 2 (t) dz 


since the solution of 


y(t)=y 0 e ' + 

For simplicity take y 0 = 0 


y+y=z(t),t> 0 

r (t ' T) z (t)dt 


y(t) = s 


-(<-T) 


x (t) dr —K 


-(* T )y 2 ( T )(dT (1) 


Proof of Convergence 

Consider the algebraic equation 

Y=\e\X+\K\Y 2 (la) 

similar in form to (1). 

Similar series solution of this equation would give 

Y=\s\X+\K\\e\ 2 X 2 +... (2a) 

This is just the series expansion of the solution 

2 |s|| X| 

of the equation (la) and so is convergent if 4 |e| \X\ \K\ < 1 i.e. 
|*| < 1/4 i e\\K\. 

Now it can easily be shown that every term of series (1) is in absolute 
magnitude less than or equal to the corresponding term in series (1 a) 
if max \X\ (/ x ) < X in which case also 
t\> o 

max | y(t x )\<Y 

1 1>0 

Hence if 

||x|j X=max|x(( 1 )|<^jL^j-, 

then the Volterra series (1 a) is convergent and also 


max|j;(L)l< 

ti >0 


l~Vl~4M llxj] |K| 
2\e\\K\ 


This is also a statement of stability of the system. 


This is a non-linear integral equation which can be solved in power 
series, e.g., by successive approximation. 

First approximation: 


y(t)=s 


i: 

—- app 
,=e £ 


e v ‘ ’ y x(r)dr4- ... 


Second approximation: 

)'(0 


(t_r) x(T)dr + 


0 

Ks 2 


-(f-T) 


-(t Tf) . 


*( T i)dTj[ ) dr +... (2) 


This is the Volterra series. 


R. H. Flake, in reply 

I want to thank Dr. Barrett for suggesting his interesting alternate 
method for calculating the Volterra series. The related investigation of 
the convergence of the series is certainly important. However, I am not 
sure that I agree with the statement that the suggested method, when 
applicable, will give the Volterra series with little calculation, since this 
type of computation will generally become very laborious after only a 
few iterations. In Reference 9, the kernels in the Volterra series solution 
for this example are calculated using the simple procedure suggested in 
this paper. I believe an examination of the two techniques applied to 
the same example will indicate that the procedure discussed in the 
second half of the paper is computationally much less involved than 
the suggested iterative procedure. 
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P. Alper, Elec. Lab., T.H . Kanaalweg Z B, Delft, Holland 

In this excellent paper, and in his previous one dealing with non-linear 
differential equations with constant coefficients, Dr. Flake has presented 
a very straightforward and powerful procedure for treating non-linear 
systems. There are several comments, however, that I wish to make 
clear concerning the applicability of the technique. 

As has been pointed out, the very serious drawback of the Volterra 
series representation is the possible divergence of the series; that is, the 
differential equation may have a perfectly well-defined solution, but the 
series itself will diverge. For example, the following differential equa¬ 
tion with a constant input, c, 


jj + y + ky 2 = x(t) = c 

has a finite output for t co when 



but the Volterra series yields a finite solution when 


n 


y=A (x)= £ 

i— 1 



(t — t) x l (t) dr 


where ki(t ) are the unknown continuous functions of time. 

Let us also assume that it is possible to determine experimentally 
(at least approximately) the so-called weak differentials of the plant: 


dA [x, hJ=Um~{A iX+yht] -A [x]} 
=^0 + ?M(y=o) 


d2A IX h lt h 2 ]= lim L {dA [x+yh 2 , h^-dA [x, / ?1 ]} 

d A [x, h l9 h 2 .../2 n ] = lim — {d n 1 A [x + yh „... h t ] 
y->0 y 


Another important point to note is that only those kernels, hn 
will appear for which 

n~g+(g-l)k, k= —1,0,1,2,... 

Then, when one is trying to experimentally determine the kernels, it 
would be quite helpful to know which kernels would not be expected 
to occur and consequently a great saving of time and effort would be 
obtained. Moreover, for the case of no initial conditions, then the first- 
order kernel will be the same as the ordinary impulse response of the 
non-linear system, h(t), 

h x (t) = h(t) 

However, when the initial conditions are not zero then this may 
not be true. 

Lastly, much of Dr. Flake s work on non-linear differential equa¬ 
tions with constant coefficients has been extended to the discrete case 
and a large class of non-linear difference equations can be solved; the 
extension to discrete systems with time-varying coefficients should also 
be possible. 


R. H. Flake, in reply 

I thank Dr. Alper for his comments and would add that I have had an 
opportunity to read some of his recent work applying the Volterra 
series to the analysis of non-linear discrete systems. I think it would be 
interesting to investigate how well some of the classical design proce¬ 
dures for linear discrete systems will work for non-linear discrete 
systems, since there are strong similarities between the analysis of 
non-linear discrete systems using the Volterra series and multidimen¬ 
sional z transforms and the usual linear discrete system theory. 


R. Kulikowski, Polish Academy of Sciences, Warszawa ne Koszy- 
kowa 75/18, Poland 

I should like to show, in connection with the very interesting paper of 
Dr. Flake, an important application of the Volterra series to the non¬ 
linear plant identification. 

For the sake of simplicity let us assume that the unknown plant 
can be approximated by the operator 


lhis can be done by observation of the output of the plant for the 


given inputs: x(t) + y ht(t), z = 1,2,... 
small number. Since 


n, where y is a sufficiently 


dA [0,1(0] = 


k x (t) dr 


d n A [0,1 (t ),... 1 (<)] = k n ( T ) dr 

where 1 (?) is the step function, we can find experimentally the functions 


k t ( t)dr 


and then 


k i0) = (p'i(t), i = l,2,... n 

Some more general results concerning this method of non-linear 
identification were described in a paper in press. 


R. H. Flake, in reply 

I thank Professor Kulikowski for his comments on the problem of 
non-linear identification and I look forward to reading his paper on 
the subject. 


V. S. Pugachev, Institute of Automatics and Telemechanics of the 
Academy of Science ofU.S.S.R ., Moscow, I~53 , Kalanchevskaya, 
15 a , U.S.S.P. ' 

The systems representable by truncated Volterra series considered in 
Dr. Flake’s very interesting paper are the special cases of systems 
reducible to linear from the statistical point of view. Hence the methods 
of the statistical theory of linear systems can be applied to solve the 
problems of statistical analysis and synthesis of such systems. Owing 
to this fact the characterization of non-linear systems by means of 
truncated Volterra series has been previously used in problems of 
statistical studies of non-linear systems. Interesting results were 
obtained m such a way by Dr. Kulikowsky in Poland and Kichatov in 
the U.S.S.R. Other applications of the statistical theory of linear 
systems for the statistical studies of various types of systems reducible 
to linear have been developed by Dr. Lubbock in England and 
Dr. Prasad in India. 
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R. Flake, in reply 

I thank Professor Pugachev for his comments on the use of the 
Volterra series in the statistical theory of non-linear systems. My first 
contact with the concept of the Volterra series was through reading 
Professor Wiener’s book on non-linear problems in random theory. 


L. P. Sharma, Department of Chemical Engineering , University of 
Virginia , U.S.A. 

The evaluation of the kernels in the Volterra series may offer serious 
problems, either in difficulty or in accuracy. The first-order kernel 
must be known, preferably in closed form; if a closed form is not 
possible, an approximation is necessary. Since an approximate first- 
order kernel will be used to find the higher-order kernels, the error 
propagated may grow to large magnitudes in some cases. If the series 
converges well (as it certainly does in the example cited), the error 


should remain tolerable and the solutions will be good. I should like 
to hear Dr. Flake’s comments on the seriousness of these problems. 

R. H. Flake, in reply 

When the linear portion of the non-linear differential equation has 
constant coefficients, then Mr. Sharma’s comments do not apply since 
the exact closed form expression for the first-order kernel may be 
obtained by routine calculation. However, the more general situation 
(15), is, I believe, clearly indicated in the paragraph following equa¬ 
tion (30) of the paper. 

The problem of finding the approximate impulse response of linear 
differential equations with variable coefficients has been discussed in 
many papers before and is not the subject of this paper. A survey of 
these techniques and an investigation of the errors resulting when they 
are applied in calculating approximate Volterra series solutions for 
non-linear differential equations with variable coefficients would, I 
believe, be a worthy subject for further investigation. 
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The Applicability of Quasi-linear Methods to Non-linear 
Feedback Systems with Random Inputs 

H.W. SMITH 


Summary 

An approximate method is derived for assessing the importance of 
non-linear effects in feedback systems with Gaussian random inputs, 
subject to the restriction that the non-linearity be of the zero- 
memory type. It is shown that the method affords an indication of the 
applicability of quasi-linear techniques to any specific problem. The 
choice of a quasi-linear method from among those so far suggested 
and the class of problem in which it gives accurate results are recon¬ 
sidered. 

Sommaire 

On a etabli une methode approchee pour evaluer l’importance des 
effets nonlineaires dans les systemes a reaction avec des entrees 
aleatoires, gaussiennes sous la restriction que l’organe nonlineaire 
soit du type instantane. On a demontre que la methode donne une 
indication de Tapplicabilite des techniques quasi-lineaires a tout pro- 
bleme specifique. Le choix d’une methode quasi-lineaire parmi celles 
qu’on a suggerees jusqu’a present, et le genre de probleme dans 
lequel la methode donne des resultats precis, sont discutes. 

Zusammenfassung 

Eine Naherungsmethode zur Abschatzung des Einflusses von nicht- 
linearen Efifekten in Regelsystemen mit regellosen EingangsgroBen, die 
einer GauBschen Verteilung geniigen, wird abgeleitet; die Nicht- 
linearitat ist als verzogerungsfrei vorausgesetzt. Es zeigt sich, daB die 
beschriebene Methode einen Hinweis auf die Anwendung der Quasi- 
Linearisierung fur jedes spezielle Problem gibt. Die Wahl eines Ver- 
fahrens zur Quasi-Linearisierung aus den bisher vorgeschlagenen 
Methoden und die Gruppe der Probleme, fiir welche genaue Er- 
gebnisse erzielbar sind, werden betrachtet. 


Introduction 

The analysis of non-linear feedback systems with random inputs 
presents considerable difficulty, and no completely satisfactory 
method of dealing with such systems now exists. The most 
easily applied method of analysis is the quasi-linear technique 
of Booton 1 . In this technique a single-valued zero-memory 
non-linear device within a feedback loop is replaced by a linear 
amplifier whose gain is chosen so as to minimize the mean 
square of the difference between the output of the non-linear 
device and the output of the linear model which replaces it. 
The gain of the linear model is a function of the amplitude 
distribution of the random signal at the input to the non-linearity. 
If this is assumed to be Gaussian of known mean square value, 
the linear model is known; linear analysis then allows the 
determination of the mean square system input of specified 
spectral characteristics which corresponds to the assumed input 
to the non-linearity. By repetition, a curve of mean square 
input against mean square error is found. 

It is apparent that this method neglects a component of the 
output of the actual non-linear element. Kazakov 2 , Axelby 3 and 


Pupkov 4 have proposed somewhat more complex linear models 
which purport to account for this component; in both these 
models, the assumption of gaussian distribution at the input 
to the non-linear device is retained. 

The statistical properties implied by the three models are: 


Booton: 


x(t)y'(t+T ) =x(t)y(t+x) 

(1) 

Kazakov/Axelby: 


C/(0] 2 =l>(0] 2 

(2) 

Pupkov: 


y’(t)y'(t+r)=y(t)y(.t+T) 

(3) 


where * (t) is the input and y (t) the output of the non-linear 
device, and y r (t) the corresponding output of the model. The 
bar denotes the ensemble average. 

It is not possible, in the present state of knowledge, to choose 
with confidence the best of these methods, nor can the range of 
applicability of any of them be determined. This paper establishes 
a basis for choice, gives a method of assessing the accuracy of 
quasi-linear methods in any specific application, and considers 
briefly the type of problem to which such methods can effectively 
be applied. 

The Functional Representation of Non-linear Systems 

The functional representation of non-linear systems of 
Wiener 5 is a powerful analytical method which rests on a firm 
theoretical basis. However, as Zames 6 has shown, its direct 
application to feedback systems leads to descriptions of signals 
which occur within the system in the form of infinite functional 
series, and serious convergence difficulties arise. At present, 
therefore, the representation can only be applied to zero- 
memory non-linear systems which are sufficiently close to linear to 
allow the convergence of the functional series to be demonstrated; 
in practice, the restriction is somewhat more stringent, as 
convergence must be sufficiently rapid to allow solutions to be 
obtained without excessive labour. 

Quasi-linear methods, although not so far justified on 
theoretical grounds, yield results of moderate accuracy for 
systems containing violent non-linearities under many conditions 
of practical interest. The distinctive feature of quasi-linear 
methods is their consideration of the relations between statistical 
descriptions of signals, rather than between signals themselves. 
This suggests that a functional representation in terms of signal 
statistics might be more informative than the direct representa¬ 
tion in terms of signals. The author has shown elsewhere 7 that, 
if a non-linear feedback system is stable, and if the input to it is 
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a Gaussian random signal, then: (a) the product of the mean 
values of any two signals appearing within the system can be 
expressed as an infinite power series with input-dependent 
coefficients and the mean value of the input as variable, and 
(b) the zero-mean second-order cross-correlation function 
between any two such signals can be expressed as an infinite 
functional series with input-dependent kernels and the input 
auto-correlation function as variable. This relation allows the 
results of previous investigations of the functional representation 
to be applied here. 

Application to a Zero-mean System 

Consider the elementary system shown in Figure 1, which 
consists of a non-linear element N and a linear element G in a 
simple feedback loop. Let the system be subjected to an input 
signal x (f) having zero mean. 


e (t) Nonlinear w (t) Linear 
£ y-—» element —* element 
V 7 N G 


C1,) 

H t (0=exp^-(-D l ^exp(-y) (l2) 

and x, a are the mean and standard deviation respectively of the 
input signal e(t). K x is Booton’s ‘equivalent gain’. . 

For the present purpose, a symmetric non-linearity is 
assumed, so that all signals within the system of Figure 1 have 
zero mean. Since 

x(t)=e(t)+y(t) ( l3 ) 

cj) xs CO = 4>ee 00 + ^ey <0 + ^ye (0 + 4yt <0 ( 14 ) 

Applying the results (4) to (12), one obtains the operator 
equation 

(j> xx (t) = (1 + K l G)* (1 + Kj})* 

+ f (15) 


Figure L Elementary non-linear feedback system 

To form a quasi-functional series describing the statistics 
of the error signal e (0, one must first determine the effect of 
linear and non-linear operations on statistical quantities. 
If G is a linear operator such that 

y(t) = G*w(t)= f /i(r)w(f-t)dT (4) 


LUV-aJL 

K y (t)= f°° h (T) 4 WW (t- T) dT= G* <^ ww (t) 

J - 00 

<j> w (t)= f h(T)<j> ww (j + T)dT = G* <^ ww (t) 


r oo 00 

(t) = h (7\) h (T 2 )cj) ww (T+ Tt - T 2 )d7\dT 2 


= G*G* </> ww OO 

Also if N is a zero-memory non-linear operator such that 

00 

w{t)=N*e(t)=f(e(t))= I C k (e(t)f (8) 

k — 0 

and if e (t) is Gaussian, then it has been shown 8 that power series 
relations hold between input and output correlation functions. 
Using the notation of reference (7), 

d> ew (r)-Ke(^KMt) (9) 


4> ww 0)= E *2 !>«(<>]*= E KlP k *<fi ee v do) 

k= 0 k=z ° 


This non-linear integral equation must be solved for r - 0 if the 
mean square error and input signals are to be related. However, 
since it is a quasi-functional equation, a series solution or 
4> ee (t) in the form 

+ ••• ( 16 > 

where E t , E s , ... are general functional operators, may, as 
Zames 6 has shown, be divergent. In fact, investigation of this 
series solution shows’, that, at r = 0, the solution is an alterna - 
ing infinite series, and that the ratio of successive terms involves, 
among other factors, the ratio KJK V If this ratio is large, the 
functional solution will fail to converge; this failure of conver¬ 
gence does in fact occur for systems of practical interest. 

An Approximate Solution Method 

An approximate solution of the integral equation can be 
obtained by replacing the terms of the form Pf <p e , W by 
white noise approximations 2 N'f. 8 (r), where N k is a constan 
so chosen as to obtain the best match to the exact terms over 
the range where they affect the solution. For clarity, this will 
be discussed with reference to the third-order P 3 . The extension 
will be clear. In Figure 1 , 

^ e (t) = (l + X 1 G)- 1 *(l + K 1 G)- 1 ^ 

[>«(t)-G*G* i^P 3 *<MT>] ( 17 > 

approximately. The equivalent linear closed-loop transfer 
operator is now defined as 


T=K 1 G*(.l+K l G)~ l 
Rewriting (17) with the aid of (18) 

k\ 4ee 00= 1 *g 1 *r* r* </> xx ( T ) 

-f* T*KjP 3 *cj) ee (T) 
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and, Fourier transforming, 



(20) 

where 



&iV(s)=K 2 

Lfiee CO] 3 exp ( — sr) dr 

~ OO 

(21) 

Now for all practical control systems, T (s) is a band- 
transfer function such that 

■limited 

T ( -jco ) T (jco) ss constant, |©| < \co c \ 

(22) 

T (-jco) T(jco)xkco~ 2n , n > 1, |©| > |coJ 

(23) 


where co c is an angular frequency to be called the equivalent 
linear cut-off frequency of the system. Thus, if the white noise 
approximation holds over the frequency range I co I < I co I 
little error will result. ~ e <■. 


are small in amplitude compared to the first, by virtue of the 
broad peaking of <P„ (s), and (b) the first loop of the function 
extends over a time range inversely proportional to the peaking 
frequency co ° 
When the operation 

J°3*^(t)=[<Mt )] 3 (24) 

is performed, the resultant function has the form shown in 
Figure 2 (b), and the corresponding power density spectrum is 
shown in Figure 2 (c). This spectrum is essentially flat for 
I <° I ^ I <“01» and may be approximated over that range by 
a white noise having the spectrum 

<£< 3) (s)«4>< 3) ( 0) (25) 

Using this approximation, the integral eqn (17) can be solved. 




Jf ®xx (?) represents an input which does not have peaks in 
its frequency spectrum, and if the system defined by T(s) does 
not exhibit serious resonant effects, then <P ee (s) is a smoothly 
varying function free from line-like spectral components. Also 
since it is generally true that feedback systems exhibit little error 
tor inputs at frequencies below their cut-off frequencies, much 
ot the energy m the spectrum of e (t ) lies in the frequency range 

beyond co Thus, m many systems, 0 ee (s) has the form shown 
in Figure 2 (a). 


v 7N COrreSponding a uto-correlation function <b e . M is 
sketched m Figure 2 (6). Note that (a) all loops of the function 


Numerical Example 

To illustrate the solution procedure, consider the system 
of Figure 1 , with 

G(s)=y (26) 


and 

Then 


$**(«) = 


2 B 2 


s(-s) 
N*e=e i 


(27) 

(28) 




2 £ 2 1 

(s + K^f-s + Kj)~(s + Kiji-s+Y,)^^ (29 ) 


First replace (s) by the white noise 2 N 2 , where N is a 
constant as yet undetermined. 


& ee (s)& 


2 (B 2 -N 2 ) 

(s + KOi-s+KO 


(30) 


Having found & ee (s), determine (s) by carrying out the 
operation of eqn (21). The mathematical techniques of George 9 
frequently ease the labour of this step. In this case. 


<t>ee(0 = 


b 2 -n 2 


exp (~K 1 |t|) 


(31) 


[<^eC0] 3 = 



exp (-3 K x |t|) 


(32) 


Now set 



6K t 

(s + 3 i^) (— 5 + 3Ki) 


2 N 2 = 0 (3) (O) 

yielding 

at*- M (£zHl ) 3 
3 *A Kx ) 


(33) 

(34) 

(35) 


It is now possible to determine e 2 (f), using ParsevaPs integral 
and eqn (33), to obtain 


e 2 (t) = 


B 2 -N 2 

Kx 


(36) 
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Up to this point, the solution has been perfectly general. and 
.Kg are now determined using (11), (12) and (28). 




1 


(<r^) 3 -exp( —— )d^ = 3ff 2 —3e 2 (t) (37) 
<7(271)* ' V 2 




1 


< 7 3 ( 127 t)* J 


(cr0 3 (^- 3 ^ ex p(-T N ) d ^ == V 6 ( 38 ) 


To obtain the relation between e 2 (t) and the magnitude of B, 
(35) and (36) must be solved simultaneously. This problem 
reduces to determining the real roots of a single cubic equation. 
In the example at hand 


and 


B 2 = K x + N 2 = 3-67 [e 2 (f)] 2 


Thus, finally 


e 2 (0 = 0-522.8 


(39) 

(40) 

(41) 


In the more general case, such a simple analytic solution isnot 
possible, and numerical calculation for each value of e- (t) 

assumed is necessary. . . , 

It will be seen from this example that a solution is obtained 

if, for all positive values of e 2 (0 assumed, a real positive value 
of mean square input results from the solution of the cubic 
equation. It is possible to show 7 , by a simple reductio ad absurdum 
proof that a valid solution is always possible using the approxi¬ 
mation made above. It may further be shown that, if the spectra 
of distortion terms arising in a functional senes solution are 
approximated by white noise in a manner similar to that used 
above, the functional series solution and the approximate 
solution are identical whenever the functional series converges 
In addition to guaranteeing a valid (though approximate) 
solution to the non-linear integral equation for system error, the 
method illustrated above lends itself well to computation. All 
terms of the functional series which result from the presence of 
a third-order term in the non-linear characteristic are accounted 
for by the solution of a single cubic equation, whereas a func¬ 
tional series solution (if slowly convergent) requires the com¬ 
putation of many spectral terms, followed by the solution of a 
high-order algebraic equation to obtain the input corresponding 
to a postulated error. Only in the case of weakly non-lmear 
systems is the labour of computation comparable. 


The Gaussian Assumption 

In deriving the error equation, the assumption was made 
that the input signal to the non-linear element xs Gaussian. 
However, it has been shown 10 - 11 that, in cases where the amp 
tude distribution of this signal can be exactly determined, this 
assumption does not hold. The value of the technique described 
above thus lies in its estimation of the importance of non-linear 
distortion rather than its improved accuracy; whenever the 
correction to the quasi-linear solution resulting from considera¬ 
tion of distortion effects is large, the Gaussian assumption fails 
Thus the method affords a rapid indication of the applicability 
of quasi-linear methods in any specific instance. 
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Figure 3. Relation between predicted and measured 
errors in simulated non-linear systems 

To demonstrate the validity of the method, non-lineai 
systems of first and second order containing cubic, relay, and 
saturation non-linearities have been simulated with random 
input signals of varying bandwidths applied 7 . A total of 108 
simultaneous measurements of mean square input, mean square 
error, and amplitude distribution of the error signal were made 
It was found that, whenever third-order distortion effects (as 
determined by the approximate solution method) were large, 

quasi-linear analysis gave results which were appreciably in error 

(F i SLlre 3 ). Moreover, where large inaccuracies occurred, the 
measured distribution of error signal was markedly non-Gaussian 

(F * It may therefore be said that, whenever the effects of non¬ 
linear distortion are of appreciable importance, any method ot 
analysis based on the assumption that the signal at the input 
to the non-linear element in a feedback system is Gaussian will 
fail to give accurate results. It follows, therefore, that there is no 
advantage in applying quasi-linear methods more eomptac than 
that of Booton 1 in instances where quasi-linear methods are 
applicable. This is equivalent to ignoring the non-linear term 
in eqn (15) above. There is some reason to believe that, when 
the errors of this simple quasi-linear analysis are large, the 
methods proposed by Axelby 3 and Pupkov* are subject to even 
larger errors. 

The Applicability of Quasi-linear Analysis 

The best available simple method of analysis for feedback 
systems containing a single zero-memory non-linearity, and 
subjected to. a Gaussian random input, is that of Booton (an 
its extension to non-zero mean systems by Somerville and 
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e-volts 


(c) 

Figure 4. (a) Input-error curves for a typical simulated system 
lb) Distribution histogram of 10,000 samples 
of error signal 

(c) Distribution histogram of 10,000 samples 
of error signal 


estimation of the mean square value of any signal within more 
complex loops 7 . However, as the method proposed is only a test 
lor the applicability of quasi-linear analysis, examination of the 
signal at the input to the non-linearity will usually suffice. 

Reflection shows that the quasi-linear method will afford 
adequate accuracy when the signal at the input to the non¬ 
linear element has the greater part of its power in a frequency 
range outside the effective bandwidth of the system. In this case, 
the distortion term in the error eqn (19) will be small. Thus 
quasi-linear methods can be effectively applied to systems 
subjected to narrow-band signals contaminated by wide-band 
noise. An extension of linear optimization theory to the non¬ 
linear case, and a method of determining the response of non¬ 
linear feedback systems to slowly varying systematic inputs in 
T 5 ^ es ® nce of , noise > have been described elsewhere 7 . Caution 
should be exercised, however, in the interpretation of results of 

qU fi S1 i near analysis even when mean square signals are predicted 
well. This does not indicate that other weightings which empha¬ 
size extreme values differently will be equally well predicted. 

Where quasi-linear methods are shown by test to be of 
doubtful accuracy, or where it is desired to use error weighting 
other than mean square, a method of non-linear analysis which 
does not make the Gaussian asumption is required. Recent work 
by Sawaragi and Sugai 13 suggests a possible approach to such 
a method by the use of cumulant functions. However, the same 
convergence and approximation problems apply to this method, 
since they are in fact inherent in the functional description of 
the system, and much remains to be done before a practicable 
engineering method of analysis can result. 


herton ). The method of analysis presented above allows 
applicability of the quasi-linear method to any specific cas< 
e determined. When, however, the predicted effect of 
non-linear distortion neglected in quasi-linear analysis is la: 
he Gaussian assumption fails, and no method dependent ui 
it can give accurate results. 

_• T{“. S pap ? r d ® a J s with the estimation of mean square er 
signal m a simple loop. The method can be extended to all 
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DISCUSSION 


C. H. P. Brookes, Department of Engineering Science, Oxford Uni¬ 
versity, Oxford, England 

The problem of deciding on the accuracy of Booton’s quasi-linear 
approximation for feedback systems containing non-linear elements 
can also be considered in this way. It is easy to extend this method to 
obtain information about the size of a simple non-linearity present in 
the system 1 . 

For a linear system with input x(t) and output y(t), 


<P X y(s)-H(s)$ xx (s) 

and 

$ yy (s) = \H(s)\ 2 <P xx (s) 

where H(s) is the transfer function and ® xx (s) etc. the power spectra. 

Eliminating H(s) from these two equations a spectral ratio R(s) 
can be obtained which is always unity. 


*«(g) •*» (*) 

l<M s )l 


= i 


' (i) 


Consider a system such as in Figure 1 of Dr. Smiths paper with 
H{s) as the transfer function of the linear element. The output spectrum 
may be expressed as a power series expansion, 


00 

<P w { S )=^a 2 n \H(s)\ 2 ^lM 

Jl = 1 

where a n are the coefficients described by Barrett and Coales (Ref¬ 
erence 8 of the paper) and 0 N ee (s) is the Fourier transform of the 
> ? th power of the normalized error autocorrelation function. 

If the non-linearity equivalent gain is K and the error R.M.S. 
value is a e then, 

a i = a e K 

and higher order terms represent the distortion components caused 
by the non-linearity. 

In addition, quasi-linear analysis gives 

$ £J ( s ) = K/f(s)# ee (s) 

and from (1) 

r , , . a\ <P ( n 2 L 0) + a\ (s)■+ • • • 

R(s)=1 + - mrjD 

= 1 +<50) 

The value of R(s) given by (2) will indicate whether or not a quasi- 
linear analysis of the system is justified because the component of the 


( 2 ) 

(3) 


non-linearity output which has been ignored will show up as a positive 
quantity added to the spectral ratio. This also applies to more com¬ 
plex systems with any type of non-linearity. 

For a simple type of non-linear element the coefficients an in the 
output spectrum expansion can be calculated and often this expansion 
converges quite rapidly. Then the value of (5 (s’) obtained from (3) will 
enable an estimate of the size of the non-linearity to be made. 

In the case of saturation, the even coefficients a 2 , a i9 etc. disappear 
and 

<5 (s)' («)'( s ) 

approximately. . 

The coefficients aljK 2 and a^/K 2 can be computed m terms or 
the ratio a e \h where h is the saturation level. Using these values and 
experimental results for S(s), ^e&is), and <$$1 (s) a solution 

for a e \h may be found. It is not necessary to know the value of H{s). 

The restriction that the input to the non-linearity must have an 
approximately Gaussian distribution still applies and, of course, the 
presence of noise in the output will introduce an error by adding to 
the value of d(s). 

The spectral ratio used here is the inverse of the system coherence 
discussed by Goodman 2 . 
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W. M. Wonham, Research Institute for Advanced Studies, 7212 Bel - 
Iona Avenue, Baltimore 12, Md, U.S.A. 

The author has provided a useful empirical rule for estimating the 
accuracy of R.M.S. error calculations based on Booton’s linearization 
method. For the author’s numerical example, the exact value of ]/e 2 
can be found after solving the (equilibrium) Fokker-Planck equation, 
and is 0-681 ]/B. Booton’s method yields 0-759 f/B and the author’s 
method 0*724 ]/B. These values seem roughly consistent with the graph 
(Figure 3) although it is not quite clear how the percentage differences 
in Figure 3 have been defined. 

Y. Sawaragi, Faculty of Engineering , Kyoto University, Kyoto, Japan 

Since a full variety of the applicability depending on both non-lineari¬ 
ties and dynamic characteristics of linear elements within a feedback 
control system exists, it is necessary to acquire more quantitative 
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results in due consideration of various combinations of them. As we 
consider more various types of combinations of non-linearities and 
linear elements, the procedure presented here becomes more compli¬ 
cated. In what manner will you develop your further study circum¬ 
venting the difficulty of numerical evaluation? 

On the other hand it is very useful to directly evaluate the difference 
between the probability distribution of the system input and the 
corresponding output. We have already studied this problem by the 
method of solving the associated Fokker-Planck equation with the 
non-linear control equation. I hope that parallel references of our 
results, concerning the evaluation of the response probability distri¬ 
bution may be helpful to explore your field, for example technical 
reports of the Engineering Institute of Kyoto University Report 
Nos. 68 and 79. 

G. S. Axelby, Westinghouse Electric Corporation , 211 Coronet Drive , 
North Linthicum , Md., U.S.A. 

Dr. Smith has presented some very useful and interesting results with 
respect to the amplitude distribution of noise in systems with non-linear 
elements. I think that more papers of this nature are needed to compare 
computed and measured data and to obtain more knowledge about the 
performance of non-linear systems subjected to noisy inputs. In fact, 
for most systems, this type of input must be considered if realistic 
results are to be obtained. I would like to ask the author if, in the 
course of his research, he has extended the work which was stated in 
Reference 3 of his paper. Actually, I believe that the author may have 
misinterpreted the results and purpose of this work to which he refers 
several times. It was the purpose of the reference to develop an engi¬ 
neering technique of calculating the R.M.S. and mean value of an 
error in a closed-loop system with a non-linear element. This ap¬ 
proximate technique was based on a graphical representation of the 
amplitude distribution. The formula of eqn (2) of Dr. Smith’s paper 
was not used directly, or implied; instead the mean and R.M.S. values 
of the non-linear element were calculated from the first and second 
moments of the output amplitude distribution as if they were the 
centre of gravity and radius of gyration of the amplitude distribution. 
Thus the method was not restricted to a Gaussian distribution as 
implied in the paper. In fact, the use of moments for the cases con¬ 
sidered gave results more nearly resembling exact computer results 
than did other methods of calculating the R.M.S. value including that 
of Booton. 

An example was given in Reference 3 where a Gaussian input distri¬ 
bution was assumed to exist with the realization, from previous work, 
that this was an approximation. The data in Dr. Smith’s paper indicates 
how this assumption may be modified to make a more accurate ap¬ 
proximation. However, the work in Reference 3 was concerned with 
the mechanism of a deterministic input as well as a random input, and 
it was demonstrated with a graphical solution that the error in a simple 
first-order system with a saturating element is dependent on the 
magnitude of the input noise and on the type of input. For example, 
it was shown that the error would grow without bound if the noise 
input became too large with respect to the magnitude of a ramp input 
for which the system would normally have a bounded error. 

Dr. Smith indicates in his paper that he has extended his work to 
include the response of non-linear feedback systems to slowly varying 
systematic inputs in the presence of noise. I assume that this may also 
be an extension of Reference 3, and I would like to know if Dr. Smith 
would comment on this extension of his work possibly pointing out 
some phenomena of interest. 


J. C. West, Electrical Engineering Department , Queen's University of 
Belfast , Ireland 

I cannot agree with the conclusions of Dr. Smith when he says there is 
no advantage in using methods more complex than that of Dr. Booton. 
Too much emphasis is placed on the importance of the amplitude 
probability distribution. It is of more value to attempt to evaluate the 


frequency spectrum of the error, especially where distortion produces 
power at low frequencies in the working band of the system. In 
Dr. Smith’s method he makes the assumption that the distortion has 
a white noise spectrum. This is a good approximation for the system’s 
working band and the determination of its magnitude is of extreme 
importance. It seems to me unimportant that the error does not turn 
out to be Gaussian. Contrary to the author, I consider his results in 
Figure 4 to show that a Gaussian approximation is a good engineering 
strategy. In fact, the input signal in practice is no better than an 
approximation. 

A Booton linearization leads to an error spectrum approaching 
zero at zero frequency and this is contrary to practical results. The low 
frequency distortions as we showed in 1958 1 is probably the most 
important requirement of the designer. 
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G. S. Axelby 

In response to Dr. West’s remarks, it would seem that the importance 
of the amplitude or the frequency distribution of the noise input to a 
non-linearity depends upon the character of the non-linearity and the 
system in which it is used. For example, knowledge of the amplitude 
distribution of a signal is most important if it is to be applied to a relay 
or to a device with a dead zone, because if the R.M.S. value of the 
signal is sufficiently small with respect to the dead zone, the relay 
would essentially remain inoperative regardless of its frequency 
distribution. 


H. W. Smith, in reply 

Before replying to Professor Sawaragi’s question, I should like to 
acknowledge the thorough work of his group at Kyoto University on 
many problems arising in non-linear systems with random inputs: the 
papeF 3 referred to is the most recent of a series of valuable contri¬ 
butions. 

As regards computational difficulty, T feel that, in general, too 
little heed is paid in this field to the development of good notations 
and computation techniques. I have found the operator algebra and 
transform technique developed in my more lengthy work, which 
extends the work of George 9 , to be much more convenient than 
manipulation of convolution integrals. Also presented in that work 
is a numerical approximation method for calculating the constants K k 
defined in eqn (11). This method allows rapid evaluation of the con¬ 
stants either by digital machine methods or by using tabulated func¬ 
tions and a desk calculator. The most laborious part of the calculation 
is, in high-order systems, the evaluation of the mean square error 
using Parseval s integral, a problem also encountered in linear systems. 
When, however, fourth-order statistics are considered, as suggested in 
my last paragraph, the computational difficulties may become con¬ 
siderable. 

Mr. Brookes’ method for the experimental evaluation of the type 
of system considered here is valid, since (p ee (s) is experimentally known. 
However, even though his spectral ratio is formally independent of the 
forward path transfer function B(s), the low-pass nature of this 
element in many systems may make experimental measurements of 
sufficient accuracy difficult. If my contention, that important non¬ 
linear effects are accompanied by a change of amplitude distribution 
of the signal e(t), is accepted, a more sensitive test for essential non¬ 
linearity within the controlled plant might be the measurement of the 
amplitude probability distribution at its input. Both methods are, of 
course, sensitive to the actual location of the non-linearity within the 
plant, and require somewhat long measurement times; data reduction 
is, however, considerably easier if distribution is used as the criterion. 
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I apologize to Mr. Axelby if, in compressing the material presented, 

I have inadvertently given an erroneous impression of his work. As he 
states, the method of linearization he used is applicable to any input 
distribution, not just the Gaussian. However, I believe I am correct in 
saying that the mean and second moment equivalence properties 
resulting from his method of calculation are identical to those resulting 
from Kazakov’s first method 2 , and that his work does imply (although 
it does not explicitly state ) my eqn (2). 

The method of analysis of systems with both systematic inputs and 
wideband noise referred to in the paper is a generalization of the work 
of Somerville and Atherton 12 and Axelby 3 . 

For such systems, it is shown 7 that the single system may be 
reduced to two independant systems: (a) a system incorporating a 
non-linearity different from that in the actual system, with the noise- 
free systematic signal as input, and ( b ) a linear system with quasi-time- 
varying gain and an input of noise alone. Solution of the first yields 
mean values of signals within the system, while solution of the second 
yields variances. A suitable rapid calculation technique similar to that 
mentioned earlier is also derived. 

In verifying the validity of the method experimentally, the increase 
in error to ramp inputs remarked upon by Mr. Axelby was predicted 
and found. 

The direct solution of the Fokker-Planck equation for the system 
used as an example is, as Dr. Wonham says, possible. (This is why it 
was chosen in the first instance.) The figures he quotes and the theoreti¬ 
cally correct amplitude probability density function of e(t), will be 
found in the dissertation which this paper partially summarizes 7 . In 
defining percentage differences, differences between the stated quan¬ 


tities were taken in the sense indicated, i.e. first quantity minus second, 
and normalized with respect to the quasi-linear predicted value. 

With regard to Professor West’s remarks, I feel that, to some 
extent at least, we are making much the same statement from different 
points of view. Effects on the frequency spectrum and amplitude 
distribution of error caused by non-linear distortion signal components 
go hand in hand. However, it is not difficult to show that (if the 
Gaussian assumption is made) these distortion components, although 
uncorrelated with e (t) in Figure i, are correlated with x {t). Thus any 
appreciable distortion component in y(t) may, and from my data 
does, have a marked effect on distribution. In my opinion, the resulting 
change of equivalent gain is a more important source of inaccuracy 

than neglect of some spectral components. 

‘Good’ engineering accuracy is also a matter of opinion, and, 
because of the logarithmic scale, Figure 4 may be somewhat misleading. 
As indicated in Figure 3, errors of 25 per cent were found in some 
systems simulated. My concern was to determine under what condi¬ 
tions the accuracy of analysis is comparable to the accuracy of 
carefully made measurements of mean square value (say 5 to 10 per 
cent). I agree that under many circumstances, input spectral data is 
not so accurately known; however, surely this strengthens the case 
for using the simplest available method under such circumstances, so 
long as it is capable of prediction within the accuracy of the data. 

In conclusion, it is well to remember that, as Dr. Wonham points 
out, the quasi-linear methods dealt with here are still largely empirically 
based and that interpretation of results will remain a matter in which 
difference in opinion is not only possible, but to be expected, until a 
better theoretical basis for the methods can be found. 
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A Digital Procedure for the Study of Non-linear Systems 

for Random Processes 

T. PRASAD and V. P. SINHA 


Summary 


Study of Random Processes 


This paper discusses the applicability of digital techniques to various 
optimization problems and not only demonstrates the utility of these 
techniques, but confirms that digital computers may be used for the 
study of plant behaviour. 

Results discussed in the early part of the paper show that analytical 
methods for the classes of problems dealt with involve mathematically 
intractable expressions; the analogue computational procedures also 
do not yield satisfactory results. 

The explicit digital relationships and the corresponding systems 
and structures developed in the paper overcome these difficulties. The 
digital methods are expected to prove most useful because of the 
availability of high-speed digital computers capable of handling 
increasingly large amounts of data. Ideas for further investigations 
are included. 


Sommalre 

Ce rapport etudie la possibilite d’appliques les techniques du calcul 
num6rique automatique a la resolution des problemes varies d’opti- 
malisation, et met en evidence non seulement Tutilite de ces techni¬ 
ques, mais egalement l’aptitude d*un calculates numerique k simuler 
le comportement d’une installation. 

Les resultats presentes dans la premiere partie de ce rapport 
montrent que les mdthodes analytiques conduisent dans certaines 
classes de problemes a des expressions mathdmatiques qui ne peuvent 
etre traitees; les m^thodes du calcul analogique ne donnent pas, elles 
non plus, des resultats satisfaisants. 

Ces difficultes sont surmontees au moyen des relations numeriques 
explicites et des systemes correspondants exposes dans ce rapport. Les 
methodes numeriques ouvrent des perspectives interessantes grace k 
l’aptitude des calculateurs numeriques a grande vitesse a traiter un 
nombre toujours croissant d’informations numeriques. Certaines 
suggestions sont presentees pour de plus amples etudes dans ce 
domaine. 


Zusammenfassung 

Eine betrachtliche Anzahl von theoretischen Arbeiten und experimen- 
tellen Untersuchungen wurde durchgeffihrt, urn die Zufallsprozesse 
(regellose Vorgange) im Hinblick auf Probleme der automatischen 
Systeme und der Regelungstechnik zu klassifizieren sowie die Unter- 
suchung von nichtlinearen Systemen bei regellosen Vorgangen durch- 
zufiihren. Die hierbei angewandten analytischen oder Analogrechen- 
verfahren fiihren zu simultanen Integralgleichungen, deren Losungen 
sehr mfihselig und zeitraubend sind. Dieser Beilrag gibt einen kurzen 
Uberblick fiber die wichtigsten bisherigen Arbeiten auf diesem Gebiet 
und empfiehlt die Verwendung der von Prasad entwickelten digitalen 
Analyse ffir die Untersuchung und Optimierung einer Klasse nicht- 
linearer Systeme. Einige digitale Systeme mit NichtlineaUtfifen erster 
Ordnung, die auf diesen digitalen Verfahren beruhen sowie einige 
zugehorige theoretische und experimentelle Gesichtspunkte zur 
weiteren Untersuchung werden angegeben. Es ist denkbar, daB sich 
derartige Verfahren nicht nur zur Untersuchung nichtlinearer Systeme 
erster Ordnung eignen, sondern auch auf eine systematische Unter¬ 
suchung von Erscheinungen bei regellosen Signalen fiihren. 


The classification of random processes on the basis of their sta¬ 
tistical characteristics and evaluation of mathematically trac¬ 
table analytical expressions for their correlation properties or 
probability distribution functions have baffled a number of 
research workers over the last few years. A fair amount of pro¬ 
gress has nevertheless been recorded. Some important contri¬ 
butions have been made by Bussgang 8 , Brown 7 , Nuttal 18 ’ 19 and 
Lubbock 14 . Bussgang has shown that for two Gaussian signals, 
the cross-correlation function taken after one of them has under¬ 
gone non-linear amplitude distortion is identical with the cross¬ 
correlation function taken before the distortion. He has sup¬ 
plemented his theoretical deductions with a number of examples 
dealing with various types of non-linear distortion. Nuttal has 
extended the work of Bussgang and has defined a general class 
of separable processes. The random processes belonging to this 
class are characterized by second-order probability density 
functions that satisfy the equation 


P (Ai> h j x 2> ti) — Pi (*1, ^i) P 2 ( x 2? ^ 2 ) 

00 

x Z a n (t u t 2 )9 ( n 1) (x u t 1 )9 < n 2) (x 2 ,t 2 ) 


where 


and 


n~0 


(0 


f 00 

TOlA) I P (^1. Y s X 2i ^ 2 ) dx 2 

J — 00 


p(x 2 , hY- 


p(x u t i ;x 2 ,t 2 )dx 1 


Pi ( X U f l)^m ) ^B 1) dx 1 =<5 m „ 

J - 00 

P 2 & 2 , h)0^ ) e ( n 2) dx 2 = S mn 


The correlation functions of such processes possess several 
invariance properties which may be utilized to advantage 18 . 
Lubbock’s work deals with a still more general class of semi- 
separable processes. Two processes x, and x 2 belonging to this 
class are semi-separable with respect to each other if: 


^ 2 ) Si (^i) Y') S2^i> 1 2 ) for t, > t 2 

where 


gi(x 1 ;t 1 ,t 2 ) = 



(x 2 wi)p 2 (xj, x 2 ;.fj, t 2 ) dx 2 


( 2 ) 

(3) 


m being the ensemble average and p 2 (x v x 2 ; t v t 2 ) being the 
joint probability density function of x 1 and x 2 . 

The invariance properties of these are the same as those of 
the separable processes of Nuttal over the restricted range 
h ^ t 2 or 4 < t v 
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A DIGITAL PROCEDURE FOR THE STUDY 


OF NON-LINEAR SYSTEMS FOR RANDOM PROCESSES 


Study of Non-linear Systems 

Almost simultaneously with the classification work, the study 
of non-linear systems for random processes, particularly with 
reference to automation and control problems, has led to con¬ 
siderable theoretical work and experimental investigations. 
Singleton 22 was perhaps the first to study non-linear filters for 
random processes. His work embodies: (a) characterization of 
the input in terms of its past values by samples at the instants 
t t — T, ... t - (N - 1 ) 71 , and (b) specification of a system 
operator F, the input-output relation being 

y(t) = F[x(t),x(t-T),...x(t-N-lT)\ ] (4) 

For determining the optimum filter, higher order correlation 
functions of the input and higher order cross-correlation func¬ 
tions between the input and output are requited. 

The works of Booton, Mathews and Siefert 4 - outline a 
method of evaluating non-linear characteristics of servo-mecha¬ 
nisms from their responses to Gaussian signals. The non¬ 
linearity discussed by them is characterized by 

X AR =f{X AI ) (5) 

where X AR and Xai denote the instantaneous amplitudes of the 
response and input respectively. The response may be separated 
into a quasi-linear and a harmonic term as 

X AR = K &q X AI (t)+X H (t) ( 6 ) 

The optimum value of the describing function A eq for least 
mean square error is given by 

f X A! f (X A j) pi (X A i) dX A! 


X A iPi ( X AI )dX A i 


erties are describable by very simple expressions, lead to a set 
of simultaneous integral equations whose solutions are very 
tedious and time consuming. 

The non-linear correlation function ordinates and the digital 
analysis of input-output statistics developed by Prasad 20 for the 
study and optimization of Zadeh’s filters of class n x overcome 
the complexities which are encountered when analogue methods 
are used 20 ’ 21 . The utility of these techniques has been further 
confirmed by Sinha 23 who has used them in making a compara¬ 
tive study of several types of filters for separating non-Gaussian 
signals from Gaussian noise in the mean square sense. He 
obtained a random input by superimposing 1,000 samples of a 
random Gaussian noise over 1,000 samples of a given non- 
Gaussian signal. The mean square errors for the types of filters 
studied by him are reproduced in .Table 1 given below. The 
values reveal that the optimum non-linear filters with storage 
give a considerably smaller mean square error and one may 
hope to reduce the error further by increasing the number of 
channels in the multipath non-linear filters 23 . 


Table 1 


Serial 

No. 

Type of filter 

Mean square error , 
normalized with 
respect to that for 
the linear filter 

1 

Linear 

1*00 

2 

Instantaneous non-linear power 
series with the first five terms 

1*65 

3 

Instantaneous non-linear power 
series with the first ten terms 

M2 

4 

Storage non-linear filter (two fol¬ 
lowed by a suitable linear filter) 

0*85 

5 

Storage non-linear filter (three fol¬ 
lowed by a suitable linear filter) 

0*68 

..... 


A theory for the experimental determination of optimum 
time-invariant non-linear systems was developed by Bose . The 
experimental procedure outlined by him for the evaluation o 
the coefficients of the optimum filters involves the use of a large 
number of gate circuits, averaging devices etc. 

Zadeh 21 studied non-linear systems in terms of classes n n , 
n = 1, 2, 3, ..., consisting of all two poles for which the input- 
output relationship may be expressed in the form of an n-fold 


integral, 

y(t)= 


K[x(t-Ti), ...,x(t-r„);xt, ... t „] 

dz u ...,dt„ ( 8 ) 


where the characteristic function K is any real function of the 

variables x(t — a), ..., x(t — t„). 

Following Zadeh, Lubbock 15 used the Barret Lampard ex¬ 
pansion 3 to study the filters of a subclass of more com¬ 
prehensively. His results are in compact and accessible forms, 
leading to an almost routine procedure for the determination o 
the optimum weighting functions of this class. 

It is evident from the examples of Lubbock that the analytical 
procedures or analogue computational methods adopted for 
optimization of even the simplest class of non-linear filters of 
Zadeh 24 for stationary random processes whose statistical prop- 


It may be mentioned here that the study of the problem 
carried out by Sinha would have been not only extremely diffi¬ 
cult but perhaps impossible without the help of the computa¬ 
tional procedures and formulae of Prasad. 


Utilization of Digital Techniques 

The scope of application of the digital techniques of Prasad 
is not limited to the optimization of filters only. These may also 
be found extremely useful in the approximate analysis of non¬ 
linear systems for stochastic processes in general. Four different 
aspects are dealt with here. 

Equivalent Linear Structures for Non-linear Systems of First 
Order 21 

The non-linear element of Figure 1 may be approximated 
by a linear network C L . If G NL is an instantaneous non-linear 
element, it is easily shown that for the class of separable pro¬ 
cesses characterized by the equation 

<£ W/ (T) = K<^;,(T),T£ 0 (9) 

where (f) is the staircase correlation function defined by Prasad 20 . 
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Gl may be a simple attenuator of gain A. The optimum value 
of A is obtained by minimizing the mean square error 

= r.-« 27^+f 

" ( 10 ) 

For minimum error, 


<hxjx( 0 ) 


In a different situation, Gjsn, may be a storage non-linear 
element of the first order (class n x of Zadeh) as shown in Figure 2. 



Figure 1. Replacing non-linearities by linear structures 



Figure 2 


In that case, Gl may be allowed to be a linear network of im- 
pulsive response g(t). The corresponding P-response ordinates 
y n ioi minimum mean square error are given by the set of 
simultaneous equations 


tenuators A l9 A 2 , .. A M and a constant term A 0 . The minimizing 
equations reduce to a set of simultaneous algebraic equations 
that may be solved for the various A’s. 

Approximate Structures for Non-linear Elements Controlling a 
Plant 

Figure 4 shows the schematic of a plant describable by a 
known linear impulsive response g(t) controlled by a non-linear 
element G NL . It is desired to approximate G NL by a linear net¬ 
work Gl such that the mean square error between f z(t) and 
j y(t) is minimum. 



Figure 4. Equivalent structure for non-linear controller of a plant 

If Gnl is a no-memory element/ [ ], Gl may be an attenuator 
whose gain A is given by the minimizing equation 

A ~Z J Ly r y s <l>U x (\r-s\ r)=EEy r y^jJ/(|r-5| t) ( 13 ) 

s r s 

where y is the P-response ordinate of g(t). The g(t) itself may 
then be evaluated from the values of y. 

IfGmisa storage non-linear device {/ [ ] followed by w (r) }, 
a linear memory structure may be chosen as Gl. Its P-response 
ordinate ex. for minimum mean square error is given bv the 
simultaneous algebraic equations 

N co 

Z o t, o 4>My(JT)<t> ix!x ({|r-s| + j}T) 


A 

I y r <t>!x! X (\r-m\ T)= L K<j> SxSf (\r-m\ T) 

F—O i- n 


/ o u r it <l > hSr(JT)<j> SxSf ({\r-s\+j} T) 


m=0, 1,2 ,N 

The impulsive response function g(t) may then be obtained 
trom y by interpolation techniques discussed by Prasad 23 . 

Still another case of an instantaneous non-linear element 
whose approximate structure may be sought for is shown in 
Figure -?■/[] is a no-memory non-linear element operating on 
a set of input functions x 0 , J x x , J x 2 , ... f xm- The approximating 
structure is a multichannel linear network consisting of M at- 



Figure 3. Equivalent gains for several random inputs 


s=0,1,2 ,N 

where 

00 

< t > SyS?(J T )= E y(mT)y(m+jT) 

m~ 0 

Equivalent Structures for Non-linear Feedback Systems 

Figure 5 depicts the essential features of a unity feedback 
system having a non-linear element C NL . A possible equivalent 
structure (in the mean square sense) is shown in Figured in 
w ich Gjstl has been replaced by two optimum boxes F and G. 



Figure 5. Non-linear feedback staircase system 
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Figure 6. Feedback system equivalent to that of Figure 5 


For the same system output in both the cases, the mean square 
error between the outputs J z(t) and J z x (t) is given by 


?= dr —i-r r° +r {M0-K(0} 2 <l* (15) 

T 0 ->oo Z i 0“ 1 " 1 J ~To 

If Gnl is an instantaneous non-linear piece / [ ], an appro¬ 
priate set of F and G may be 

M— 1 

F= E AJtU (16) 

i— 0 

and 

G — A8(t) 

where is a known non-linear function, A and A h i = 0, 1, 

(M — 1), are unknown constants to be determined. For mini¬ 
mum mean square error, these are given by 

u -1 

E A i$SySfi(°')- A< l > SySy(ty = ( l ) SySz^ ^ 


1VJL X 

i = 0 

for r = 0,l,2, 1 

Other modified structures, linear as well as non-linear, may 
be obtained on the same lines if Gtvl is a no-memory non-linear 
element. 


Self Optimizing Systems 

A somewhat different class of problems in which the staircase 
techniques are again useful relates to self-optimizing control 
systems. These form a topic of considerable interest at present. 
A comprehensive bibliography of research carried out in this 
field is given in the work of Aseltine et al 2 Some more recent 
publications are those of Anderson 1 , Kalman 13 and Margolis 
and Leondes 16 . 

Based upon the principles of operation, self-optimizing 
systems may be broadly classified as learning systems and 
computing systems. 

The following is confined to computing systems only. 

The block schematic of an input sensing self-optimizing or 
adaptive system is shown in Figure 7. The plant P is a time- 
invariant structure described by its staircase P-response J y{t). 
The quantities 

<M;T)= E y (kT)y(k+rr) 

k = 0 

are assumed to be fixed during the interval the system operates. 
The output of the plant, J y(t), is to follow a desired output 
J z(t) in the mean square sense; the input J x(t) may have slowly 


varying statistical characteristics. The plant P is preceded by a 
self-optimizing controller, SOC. It is a varying parameter net¬ 
work described by its impulsive response 

N 

w(t) = A 0 S(t)+ E A r e~ Krt (18) 

r = 1 

where K r is a fixed time constant and A r is a varying gain under 
the control of the signals received from £ 3 . The control unit 
consisting of E xx , E yi £ x , E 2 , E z is prescribed to set the gains of 
the SOC to such values as maintain minimum mean square 
error between J z(t) and J y(t). 

The plant output is 

jT(0 = EEE^- r y,,-fcF(t-nT) (19) 

n k r 

The optimizing equations for the system are 

E w r“| r-s\=Ps ( 20 ) 

r =0 

for j = 0 , 1 , 2, .... AT; |r - s| represents the modulus of r - s 
where 

«K= E <^ V 0’ T )^JxJ*(^+j' 7 ') 

j-o 

for all k > 0 and 

£m= E T(j T ’)0J.xtz( m +i T ) 

j = 0 

for all m > 0 

The ot and are evaluated by the computing units E 1 and E 2 
respectively. E xx , E xz and E y supply the various correlation 
ordinates required for the computation. Eqn (20) is solved for 
the staircase ordinates u from which the gains A of the SOC may 
be obtained as discussed earlier. The computer corrector E z 
solves eqns (20) in two stages and finally sets the values of the 
gains A 0 , A v A 2 , An accordingly. 



Figure 7. Self optimizing staircase control system (linear controller) 

The plant of Figure 7 has been assumed to be time invariant. 
The theory of optimization may be easily extended on the same 
lines to time-varying plants also. In this case, an additional 
computing mechanism has to be incorporated in the assembly 
in order to evaluate the varying dynamic characteristics of the 
plant in terms of P-response ordinates. This is shown in the 
sectionalized diagram of Figure 8 . 

A further generalization results if we also remove the restric¬ 
tions of structural linearity on the SOC and the plant. The 
situation, however, becomes difficult even for non-linear plants 
of class t q of Zadeh. Explicit optimizing relationships for the 
SO C in this case cannot be derived without an approximate 
linearization of the characteristics. 


Ill 
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Figure 8. A computing mechanism for plant behaviour 


Conclusions and Remarks 


The foregoing discussion throws enough light on the appli¬ 
cability of the digital techniques to various optimization prob¬ 
lems. The cases considered 20 ’ 21 not only demonstrate the utility 
of these techniques but also confirm that digital computers may 
be confidently used for the study of plant behaviour. The digital 
techniques may also lead to a systematic study of random 
phenomena. Further experimental and theoretical researches in 
this field may perhaps provide adequate foundations for com¬ 
prehensive study of adaptive and learning mechanisms, problems 
involving non-stationary stochastic processes and time-invariant 
structures, linear or non-linear. 

The results summarized in the earlier sections of this paper 
readily show that analytical methods for the classes of problems 
dealt with involve mathematically intractable expressions; the 
analogue computational procedures also do not yield satis¬ 
factory results. The explicit digital relationships and the cor¬ 
responding systems and structures discussed subsequently appear 
to overcome these difficulties. The digital methods are expected 
to prove most useful because of the availability of high-speed 
digital computers capable of handling increasingly large amounts 
of data. A few ideas for further investigations are outlined below. 

(a) A major experimental investigation constitutes the study 
of self-optimizing controllers, linear as well as non-linear, for 
slowly varying random processes described by auto-correlation 
functions of the type 


^(T) = p 1 e“^ lrI 4 -p 2 e“^ |t| -b ... +p„e 


in the stationary state. Another problem which may be pointed 
out is that of studying the systems of first order for non-stationary 
random staircase inputs by introducing known variations in 
their stationary time domain records. 


(b) The analyses presented 20 are restricted to staircase random 
functions that are synchronized with respect to a fixed closure 
time. Experimental and theoretical study of non-linear staircase 
systems may be carried out for cases in which two closure 
periods bearing integral fractional ratios to each other are in¬ 
volved. Other first order non-linear systems which contain more 
than two sampler clamps may likewise be considered for mul¬ 
tiple sampling, multirate sampling and mixed sampling 17 . 

(c) The linear and non-linear correlation functions that form 
the basis of the staircase techniques characterize the behaviour 
of the system only at the sampling instants. A rigorous or 
systematic approach for the analysis of first order systems in 
between the hold times is lacking. Perhaps a fictitious delay on 
the same lines as that proposed by Jury 11 may be introduced to 
obtain theoretical expressions giving the information between 
the sampling instants. 


(d) The staircase techniques may be applied satisfactorily to 
practical problems with any desired degree of accuracy if the 
random functions are well behaved. It is suggested, however, 
that erratic random processes may be dealt with in a better 
manner by adopting the periodic sampling procedure as pro¬ 
posed by Jury and Mullin 12 . 

(e) The ^-transform of Farman-Farmaian 9 ’ 10 for sampled- 
data control systems with finite pulse width offers a promising 
avenue for the study of the sampled non-linear systems of first 
order. Apparently, any time function multiplied by the inter- 
polatory unit pulse of Farman-Farmaian may be seen to possess 
discontinuous but periodic properties similar to those of the 
staircase function. Nothing, however, can be asserted unless 
comprehensive analytical and experimental work is carried out 
on this topic. 
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DISCUSSION 


E. I. Jury, University of California, California , US.A. 

Statistical study of non-linear discrete systems is drawing much 
attention and interest at the present time, therefore this paper con¬ 
stitutes a timely and valuable contribution in this field. In a separate 
study on a slightly different problem we were able to extend the 
statistical technique to discrete systems and specifically to pulse width 
modulated feedback systems. This work appeared in April 1962. 

Like Dr. Prasad, we found the separability property discussed by 
Dr. Nuttal to be of much use in our statistical study of linearization. 

Extensive experimental study on digital computer simulation of 
the non-linear discrete systems had verified the validity of this approx¬ 
imation; the error involved in most cases was confined to 5 per cent 
and this is a rather comforting sign for engineering work. It would be 
interesting to know if Dr. Prasad has conducted an experimental 
simulation study of the discrete system and, if so, whether the error 
of approximation is within engineering approximation. 

As a final remark, I mention that the modified 2 transform could 
be used more conveniently than the P transform. This has been carried 
out in our work 1 . 
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T. Prasad, in reply 

I thank Professor Jury for advocating the utility of digital techniques 
for the study of non-linear systems for random processes. The mathe¬ 
matical study reported in this paper appears to be important for two 
reasons; first, because of the fact that the non-linear systems discussed 
are capable of using more statistical information, and secondly because 
of the fact that the techniques reported here, and the structures pro¬ 
posed with their corresponding computational formulae, are envisaged 
to be of practical use to the engineer. Professor Pugachev has already 
referred to these approximation techniques 1 in his survey paper read 
at the 2nd Congress, Basle, 1963. 

The reply to Professor Jury’s question may be given in the affirma¬ 
tive on the basis of the work of Sinha and Naresh (Reference 23 of 
the paper), outlined. More complicated and practical experiments also 
can be simulated as suggested here; the engineers and experimenters 
from other disciplines have to undertake the task. It may be added 
that all the non-linear digital structures or systems of first-order 
discussed in this report are stable in the sense of Professor Tsypkin’s 
criterion presented at this Congress 2 . 
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Time-optimal Systems with Random 
Noise Disturbances 

V.N. NOVOSELTSEV 


Summary 

Time-optimal control systems with random noise disturbances are 
discussed in the paper. It is shown that for a class of control systems 
the controller programme which provides minimum time of the 
transient for expected values of system coordinates, assures also the 
minimum expected value of the transient time. The construction 
method of switching surfaces to provide the minimum of expected 
transient time based upon the conventional equation of optimal 
switching surface, and characteristics of different noisy channels of the 
system, is given. The examples of the second-order time-optimal 
systems with noise disturbances are considered. Results of analogue 
computer simulation of the control systems with plant transfer func¬ 
tion in the presence of random noise disturbances applied in various 
points of the system are given. 

Sommaire 

Cette communication a pour objet les systemes de reglage a temps 
optimal en presence de perturbations aleatoires constituant un ‘bruit’. 
On y montre que pour une classe de systemes, le programme de 
reglage qui, pour atteindre des points de fonctionnement predeter¬ 
mines, rend la duree du regime transitoire minimale donne en meme 
temps k ce minimum une valeur bgale au minimum de la duree predite 
du regime transitoire. On indique, pour la construction des surfaces 
de commutation, la methode qui assure le minimum de cette duree 
predite, methode qui est fondee sur l’equation habituelle de la surface 
de commutation optimale; on donne aussi les caracteristiques de 
differents canaux ‘bruyants’ au sein du systeme. On traite a titre 
d’exemple les systemes' du second ordre a temps optimal sujets a un 
‘bruit’ perturbateur. On cite enfin les resultats obtenus au cours de 
la simulation de systemes de reglage sur calculatrice analogique, en 
tenant compte de la fonction de transfer! de Installation en presence 
de perturbations du type ‘bruit’ appliquees en divers points du 
systeme. 

Zusammenfassung 

Der Aufsatz behandelt zeitoptimale Regelsysteme mit rauschformigen 
Storsignalen. Es zeigt sich, daB fur eine Klasse von Regelsystemen die 
Reglereinstellung, die die kiirzeste Ubergangszeit fiir gewunschte 
Werte der Systemparameter erzeugt, auch mit Sicherheit die kiirzeste 
Ubergangszeit fur den Ensemblemittelwert ergibt. Das angegebene 
Konstruktionsverfahren fur die Schaltflache zur Erzielung der kurze- 
sten Ubergangszeit beruht auf der tiblichen Gleichung fiir die optimale 
Schaltflache. Auch die Eigenschaften verschiedener rauschbehafteter 
Kanale des Systems werden behandelt. Zeitoptimale Systeme 2. Ord- 
nung mit rauschformigen StorgroBen dienen als Beispiele. Die Ergeb- 
nisse der Nachbildung am Analogrechner von Regelsystemen mit 
einer gegebenen Ubertragungsfunktion der Strecke und mit an ver- 
schiedenen Stellen des Systems einwirkenden rauschformigen Stor¬ 
groBen werden verglichen. 


Introduction 

This paper examines the problem of optimal control of a plant 
with constant coefficients, having one input and one output: 

x = f 1 (x) + vf 2 (x) (1) 


Here x is an ^-dimensional vector which defines the state of the 
plant, v is the control signal sent to its input. 

Functions / 1 (A) and f 2 (x) are defined and continuous for 
all x and continuously differentiable with respect to all coordi¬ 
nates of vector x: 

Xi = d l xldt l 

Equation (1) is linear with respect to v and non-linear with 
respect to x 9 and is therefore somewhat more general than the 
equation 

X = Ax J rBu 

usually considered for the case of a scalar control signal u. 


z 9 



Figure 1 


Figure 1 shows a block diagram of the system under con¬ 
sideration in the presence of interference; the following symbols 
are used: 

A —zero-memory controller 
B —controlled plant (1) 

H —zero-memory plant coordinate metering channel 
G —zero-memory control signal-to-plant channel 
Z—master-signal channel 

h, g and z -—random interference in channels H\ G and Z 
respectively with known characteristics 
u —control signal (scalar) 
v —noise-distorted control signal 
x d —true state of plant 
x n —observed state of plant 
x 2 —set point of system phase space 
x—error vector x — x z — x d 

The true-state point x d has to be shifted into some small 
vicinity of the set point (origin of the phase space). Then in the 
optimal system the following equality must be satisfied 1 ’ 2 : 

£{T[> (0) ]} = min 

The minimal value of E{T(x)} will be denoted by T* (x). 
Then in the optimal system 

£{r[x (0) ]} = T* [x (0) ] (2) 

Here x <°> is the initial value of the error vector. 
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However, considerable difficulties are involved in calculating 
the system directly from this criterion. It is far simpler to use 
the criterion of the minimum time of the transient process for the 
mathematical expectation (the minimum time required to bring 
the system to a state of statistical non-displacement) 3 ’ 4 . Usually 
considered for the determination of this time is the relationship 
£ = .E{x}, which describes the transient processes in some 
equivalent system without interference. A system in which is 
provided the minimum time of the transient process for the 
mathematical expectation T (I) = 6 (. x) 

6 (x) = min = 0* (x) (3) 

will be termed optimal with respect to the criterion 0*. A system 
in which condition (2) is satisfied, will be termed optimal with 
respect to the criterion T*. 

Functions T* (x) and 0* (x) are defined for all points of the 
phase space and T* (x) >0, 0* (x) > 0 while 

T*(x) = 0*(x) = 0 

when, and only when, the point x lies in the set vicinity of 
the origin 

zW<< 5 2 ( 4 ) 

i ~ 0 

Consideration will be given to the state of the control 
system only at discrete moments of time, as in solving similar 
problems by the dynamic programming method. For this the 
small interval of time A will be introduced and it will be con¬ 
sidered that on this interval the values of the control signal and 
the interference signals remain invariant, but at moments of 
time t = &A they change stepwise. Interference on neighbouring 
intervals will be considered independent. 

Control signal u is constrained with respect to the modulus 

\u\<N (5) 

To simplify the examination it will be assumed that both u 
and v are quantized in level with a sufficiently small pitch of 
quantization, and: 

ueQ(u) Q(u)~{u u ii 2 ,...,u r } r <oo 

veQ(v) Q(v) = {v u v 2 , l< °° 

In such a case it is convenient to describe the influence of 
random interferences in the following way. 

Since at the controller A there arrives only the value of the 
error x distorted by the noises along channels H and Z, instead 
of the correct, necessary control signal u at each moment of 
time another control signal, generally speaking distinct from 
w 0 (x< 7 ‘>), will be chosen. The probability of the choice of control 
signal u at a given moment of time, when in fact control signal 
u°, denoted by p u o u is optimal, depends in the general case 
on the position of the image point x in the system phase space. 
Thus, in a relay system this probability strongly depends on 
distance of the state point from the switching surface. When 
this distance is great, the probability of error in choice of the 
control signal is small, but as this distance is reduced even very 
weak interference can lead to error in the choice of control 
signal. If one denotes the probability of an event which consists 


in the appearance on the output of A of signal u m , whereas the 
optimal choice would be iP (x) = u z by p mh then 

PllWPl2 W —PlrW| 

11 p „ o „( x )|| = P 2 i(x)p 22 (x)...p 2r (x)\ ( 6 ) 


\p,l 00 Pr 2 ( x)--Pn 001 | 

The control signal u (x< fc) ) = u m chosen at the ktb moment by 
controller A reaches the plant along the channel with noisy G , 
where, under the influence of interference g, control signal u 
becomes control signal v. The probability of the transforma¬ 
tion of Ui into Vj will be denoted by q#. Then 



(?1 l(Z 12 * 

..q u 

tfuv II 

^21^22 * 

•* (?2 1 


1 Qr2 • 

~4rl 


Both matrices \\p u o u (x) || and \\q uv \\ can be joined into one, which 
will fully describe the action of all the interference upon the 
system: 

llp u o„|| = ||p u o„(x)||?„„|| (8) 

Matrix (8) determines for each point of the phase space the 
probability of the arrival at the input of the plant of control 
signal Vj, when Uj is the optimal signal. 

Basic Relationship for Time-optimal Systems with Interference 

In the optimization of control systems with respect to the 
criterion T* by the dynamic programming method, the following 
equation can be obtained: 

r*|> w ] = A+ min T*[x ,k+1 >] ( 9 ) 

t|(k> e Q (u) 

Here x< 7c > and u < 7c) denote the values of x and u on the /cth 
interval of time. Equation (9) is the basic relationship in solving 
such problems 2 ’ 5 . This relationship will be given another form 
more suitable for the purposes of this paper 6 . 

In open form eqn (9) is written as follows: 

T* [x (i) ]=A + min | £ p^(x w )-r*[x<‘ +1) J ( 10 ) 

(m) 0=1 J 

m = 1 , 2 ,..., r 

In the latter equation min {oc m } denotes the minimum of the 
numbers <x w , and [x< 7c+1 >] the position of the state point at the 
(k + l)th moment of time, provided that at the klb moment 
the point was in the position and at the plant input there 
arrived control signal Vj. Thus the mth term of the expression 
in brackets is simply the mathematical expectation of the time 
of the transient process when control signal u m is chosen at 
point *(*>. Averaging is performed for all the states of the system 
at the (k + l)th moment of time. The probabilities p mj in (10) 
are elements of the matrix ||p w o J, which is determined by (8). 

By introducing the sampling interval A eqn (1) can be 
rewritten in the form 

xf +1) = x\ k) + cpl+v(pf i = 0,1, 1 

xf + 1) = v (k + 1) 
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where, for brevity, is written A • f 1 = g> 1 ; A • p =cp 2 . In sub¬ 
sequent operations the relationship to xW, where possible, is 
dropped. If in the expansion of T* into a series, it is possible 
to limit ourselves (for sufficiently small A) to terms of no higher 
than the first order of smallness, then 


T* [*<* +1 )] J = T* [4 fc+ J >, x ( * +1) 


i (•••) x l-i]; 
= T*[x<* ) ,xf,...,x<*> 1 ] 

, V dT* [x (fc >] r x 

+ E -Q-- A i<Pi +Vj(Pil 

1=0 


( 11 ) 


Substituting (11) into each bracketed term in (10), after 
elementary transforms 


£ Pmi T* [x (A+x) ],= t*+" f 


j=i 


i = 0 


n y far* 2 7 
+ I‘j-foT'Vi - E PmjVj 

i=0 t j=l 


is obtained. 

The following notations will be introduced 


( 12 ) 


optimal with respect to the criterion d*, which permits the 
construction of such a sequence of control signals for any x. 
The recurrent relation (15) will now be applied to the point 

[l<* +1 >L: 

r*R'*«>].-A+min{r*R«”’]„} „_1,2 (16) 

("1, «) 

Substituting the resultant expression back into (15), one obtains: 
r*[x<*>]=2A+ min{r*[^ +2 >] m „} (17) 

(m, n) 

m = l,2,...,r; n = l,2,...,r 

similarly 

T*[x w ] = S-A+ min (T* 

( m, h, -, l) (1®) 

s 

m = l,2,...,r; n=l,2,...,r; 1 = 1,2,..., r 

One now selects m = oc, n = ft ...,/ = cr. Then, by virtue of 
condition A, the second addenda in the right-hand side of (18) 
vanishes, and, bearing in mind condition B, 


v*(x w )=E{v\u m ( x M)}= £ p mJ ( x <k) )■ vj (13) 

7=1 

B ( ‘ +1) L = £{*<* + 1) |x w ,p,t(x«)} (14) 

Here y OT * (x< fc ») is the mathematical expectation of the signal v, 
and [f< J:+1 >L the mathematical expectation of the random 
vector x( ,c+1 ) for the choice of control signal = u m at the 
point xW. Then, on the basis of (11)—(14), the initial relation 
(9) can be written in the form: 

r* [x w ]=A + min {T* m = l,2,...,r (15) 

(m) 

Control signal u m , with which the minimum is reached, is the 
optimal control signal at the point x C c ) of the system phase 
space with respect to criterion T*. 

Equation (15) is the basic relation in examination of time- 
optimal systems with noise present. 

The optimal control action at each moment of time must be 
so selected that the magnitude of neX f step is 

minimal . 


Control Algorithm in a System Optimal with Respect to 
Criterion T* 

Consideration is given to the point *<*> and the sequence 
of control actions «<*> = u a , u^ +1 ) = Ufi) ..., «(*+£) = u a under 
the influence of which the state point £ shifts consecutively from 
position xW to positions [f< 7c+1 ] a , [f(*H- 2 )] aiJ , [f( fc +* s ' ) ] a/? , 

•^ et t ^ le sequence u a , ..., u G be selected in such a way that: 

(A) The point [| r<fc+ ^] aj8 ... <3 .lies in the vicinity of the set point (4). 

(B) For all sequences u( Jc+1 \ ... u&+d) w here d < S } 
condition A is not satisfied. 

Then the sequence z/ a , Up, ..., u a is optimal with respect to the 
criterion <9*, and S is the number of steps in the optimal transient 
process with respect to the criterion 0*. It will be temporarily 
assumed that there exists and is known a control algorithm 


T*[* (/ °]=S* A (19) 

can be written. 

Thus in the case under consideration the duration of the 
transient process with respect to criteria T* and 0* is identical, 
and at each point of the phase space the control signals optimal 
with respect to 0* and T* coincide. 

The control algorithm optimal with respect to criterion 0*, 
ensures the optimality of the system with respect to criterion T* 
as well. 

Consideration is now given to the construction of algorithms 
optimal with respect to the criterion 0*. The plant studied is 
linear and is described by the equation 

n- 1 

Z a ijXj i = 0,1,...,71-1 

7 = 0 (20) 

x n = v 

It will also be assumed that the control algorithm, which 
ensures time-optimality in the absence of interference, is known 
and set in the form of a switching surface in an n-dimensional 
space: 

•A(A\ x 0 ,x it ...,x„_ 1 ) = 0 (21) 

Equation (21) contains in an explicit form the magnitude N from 
eqn (5). In the absence of interference the optimal control signal 
has the form 

u°=—N sign i jj (AT, x) (22) 

and only the values of v = u = ± N reach the plant input. 

If in the control system (20)-(21) there are noises, then in 
place of a system with interference, an equivalent system without 
interference can be considered, in which instead of the coordi¬ 
nates x(t) the relationships i(t) = E{x(t)} are considered. The 
optimal control signal in such an interference-free system will 
be optimal with respect to the criterion 6* in the initial system 
with interference, and will hence be optimal with respect to the 
criterion T*. 
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The maximum and minimum values of the signal v* which 
reaches the plant input when | u\ < N will be denoted by vm* 
and v m *. 

For the symmetrical matrices (6)-(8) vm* = ~ v m *. For 
simplicity, the examination will be confined to the case when the 
signals vm* and v m * are obtained following the selection on the 
controller of control actions u = + N and u = — N respec¬ 
tively. 

Introduced here is the coefficient of efficiency of control in 
a system with interference, which has an obvious sense: 

y(x) = ^=-^ (23) 

It is convenient to examine as an equivalent system a system: 

n— 1 

*■= Z a,jXjix n = u (24) 

) = o 

i = 0,1, —1 


Examples 

(1) The Optimal Second-order System with Noise in the Communi¬ 
cation Channel 

Consideration is given to a control system which has been 
thoroughly studied for the case of no noise; the block diagram 
of this is given in Figure 2. 

The equation of the optimal switching line of the system 
without interference has the form: 

Xo= ~2 W SignXi ( 28) 



Figure 2 


and the constraint \u \ < N is replaced by | u | <N-y(x). 

The following three cases of the action of interference upon 
the system are considered: 

(1) In the system, interference is present only in channel G. 
Here, as follows from (7), y = const., and constraint (5) is replaced 
by the constraint | u | < yN. 

The plant equation remains unchanged. The optimal 
switching surface has the form 

\lf(yN,x 0 ,x u 0=0 (25) 

(2) Interference is absent from the channel G, but the 
influence of interference h and z manifests itself in the appearance 
of additive noise along the coordinates x 0 , x l9 ..., x n - t ; at the 
controller A there arrive the values 

x* =x 0 + *7 0 

x* =x x + fh 

X n - 1 == X n - 1 tfn - 1 


It will be taken that under the influence of interference g 
the control signal is able at each moment of time to adopt 
independently one of the following values 

a t u with probability p x 


a 2 u with probability p 2 


v=\ 


(29) 


a m u with probability p m 
(u adopts the value ± N). In this case 

m 

y= £ a t p h 

i = 1 

and in accordance with (25) the equation of the optimal switch¬ 
ing line in the system with interference (29) has the form 




A 


• sign*! 


2kN £ a t pi 


(30) 


while the random component is constrained with respect to the 
modulus: 

foil <rif, /==0,1,rc — 1 (26) 

In this case it can be shown that the optimal control of 72- 
order plant with real non-positive roots follows the 77 -interval 
theorem and the switching surface has the form: 

1 / 7 (iV, x 0 + ri* (27) 

In this equation, A* = ± 1. For the second-order control 
system A* = —sign x it i — 0.1. 

Eqn (27) may be obtained by the ‘inverse motion 7 method 
(see for example ref. 7). 

(3) In the system there is present both interference in the 
channel G and unconstrained interference rj. 

In this case it is possible to construct approximately optimal 
control systems, in which the duration of the transient processes 
exceeds the minimum possible time by not more than the 
preset e. 

One such system is considered in Example (2). 


(2) The Second-order System with Noise in Channel H 

The block diagram in Figure 3 is considered. Here in the 
channel serving for metering the coordinate x x the additive 
interference rj is a Gaussian noise with zero mean value: 


PM = 


1 

-^= ex P 

<J ny j2n 



(31) 


It is required to ensure time optimality with accuracy of no 
less than 5 per cent with an aperiodic transient process. 



Figure 3 
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It is known that transient processes without overshoot in 
the system under review correspond to the switching lines 

*°^ signXl < 32 ) 

when a <Z 2, while a 5 per cent extension of the transient process 
in the processing of step signals is obtained when a = 1-650. 

In a relay control system the coefficient y (x) is determined 
by the equation 

y(x) = l~2 p H (x) (33) 

where p H (x) is the probability of wrong choice of control 
signal at the point x: 

Ph ( x ) = P {sign u° (x) = — sign u (x)} 

The magnitude of y (x) rises as the distance r t = x ln — x x 
increases, and at some becomes greater than 0*825 = 

2 * 1 * 650. Here x ln is the coordinate x x of a point lying on the 

switching line and having a coordinate x 0 identical with the 
point under consideration x 0n = x 0 : 

'I'(x 0b9 Xi„ 9 N)=*0 

For all the points x, for which r x > r x , the magnitude of y(x) 
will be replaced by y* = 0*825. The resultant control system 
with interference will possess the property, that y(x) <; 0-825 x 
for all, while for r x < r\,y < 0*825. 

The reduction of y (x) when r x < r x * stems from the presence 
of the constrained interference ry* which only manifests itself 
when r x < r x * : 

|i»*l<rj (34) 

The examination of a system of control of a plant has been 
arrived at with the equation kip 2 under the constraint 


T TTTT T T I I T I I I I I IT I T~T I T! I ITT 1 I I I T f f T T 1 




C b ) 


\u\ <0-825 N 


[in such a system the switching-line equation has just the form 
of (32)], while on the system there acts the constrained inter¬ 
ference (34). It only remains to find the magnitude of r x and 
substitute it into eqn (27). 

From the general formula 


\<J n ^J2J \G tjy J2)_ 

taking (33) into account, one obtains 

and for y* = 0-825 one has r\ = 1-343 a . 

The equation of a switching line which is optimal with 
accuracy up to 5 per cent has the form: 



P{a.<r]<^}=~ 


(x t —1-343 cr^ sign xrf 
1-650 kN 


sign x 1 


(35) 


Experimental Results 

Figures 4 and 5 show the graphs of response to step signal 
of 20 V amplitude in a system controlling a plant 1 Ip 2 , N = 20 V. 



Figure 4 corresponds to Example 1, and in Figures 4 ( b ) 
and 4 (c) y = 0-645. The switching line equation in Figures 4 {a) 
and 4 (b) has the form: 

2 

* 0= ~lo signx i ( 36 ) 
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(c) 

Figure 5 

The optimal transient process (Figure 4) is ensured in a system 
with interference following the choice of the switching line 

Xo= -Js signXl 

Figures 5(a ), (b) and (c) illustrate Example 2 for s = 4 per cent, 
y* = o*90. 5 (a) shows the performance of a step signal 

of amplitude 20 V without interference with switching line (36). 
Figure 5(b) demonstrates the performance of the signal for 


a = 14.3 V and a switching line which is optimal without 
interference. The optimal (with accuracy up to 4 per cent) 
transient process without overshoot is shown in Figure 5(c). 
The switching-line equation is 


*o= — 


(x t — 23-6 sign x t ) 2 
36 


sign*! 


The frequency band of the noise signal 9 ^ is 10 c/sec. 
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DISCUSSION 


W. M. Wonham, Research Institute for Advanced Studies, 7212 Bellona 

Avenue, Baltimore 12, M.D., U.S.A. 

If I understand the author correctly, he proposes to replace a stochastic 
control problem by a deterministic control problem which would be 
equivalent, in some approximate sense, to the original. The author 
suggests defining the ‘equivalent’ deterministic problem by using as a 
performance index ‘the criterion of the minimum time of the transient 
process for the mathematical expectation’. 

It is not clear to me precisely what the author means by this 
criterion, or by the corresponding value function T*. There is a diffi¬ 
culty here which the author may have overlooked, namely, that the 
expected trajectories of a random process cannot, in general, be identi¬ 
fied with the trajectories of some non-random dynamical system. In 


other words, one does not have an ‘equivalent deterministic system 
to optimize. 

To illustrate this, suppose that an admissible control law has been 
chosen, and that the resulting stochastic system generator is a homo¬ 
geneous Markov process [x(t)] with the transition function 1 F(t,x,F). 
The expected trajectory starting at a state x 0 is by definition: 

m (t, Xq) = j xP ( t , x 0 ) dx t> 0 

A non-random dynamical system, which is equivalent in the sense 
described, exists only if the function m has the semi-group property, 
namely 

m(s+f,x 0 ) = m[J,m(s,x 0 )], s> 0, t> 0 (1*) 
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Yet the Chapman-Kolmogorov equation 1 by no means implies the 
truth of (1*), which is generally false, except when the stochastic 
system is linear. In fact, the author’s actual procedure amounts to 
assuming that (1*) is approximately valid when s and t are small: 
that is, of the order of the sampling interval A. Yet it is precisely the 
validity of (I *) for all £ > 0, t > 0 which is necessary if dynamic pro¬ 
gramming arguments are to be relevant in this context. 

Finally, I would welcome clarification of the following points of 
detail. 

(1) If noise is present in the feedback channel, it is meaningless to 
seek a control law in the form of a function of the state. Yet the 
author seems to do just this, and indicates no filtering operation. 

(2) In the dynamic equations (20), the plant input is completely 
decoupled from all but one of the state variables. 

(3) What is the graph of the switch curve of eqn (27) ? 

(4) The result for the switching curve, eqn (30), seems difficult to 
interpret if £ aipi < 0. Yet the author nowhere states that the restric¬ 
tion £ a%pi > 0 may be necessary. 
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V. N. Novoseltsev, in reply 

The difficulties which arise when the condition 

m(s + t, x 0 ) =m[t,m (s, x 0 )] (1) 

is not satisfied, can be avoided by examining the assembly average of 
the trajectory and also the assembly average of the trajectory with 
fixed initial conditions: 

m(x, t)= j xP(x o ,0;x,T;u)dx (2) 

D(x) 

where P(x 0 , 0; *, r; u) is the probability that at a moment of time r 
the system, under the influence of control u, will be in a state x, if at 
the initial moment t 0 = 0 it was in the state x 0 . The integral is taken 
for &(x), the whole domain of variation of *. 

When in eqn (2) r rises continuously from 0 to co with a fixed 
0 < t <oo, the geometrical place of the points m(x,r) in the 
phase space is the assembly average with initial condition x 0 . 

For some u(t) let trajectory (2) first reach the specified domain 
when t = 0 (for some u(t) the quantity 6 can prove infinitely great). 
Then 0* is the minimal value of 6 , achieved when u(t ) = u°(t): 

9* —min Q[u (t)] 

u (0 e Q [u (£)] 

As shown in the paper, the control algorithm 

«°(0 = tt°[x(0] 

ensuring optimality in regard to criterion T*, i.e., ensuring the mini¬ 
mum T is also optimal in the sense of speed of response. 

Although the proof of the latter assertion is based on relationship 
(9) in the paper, which is also used in deriving Bellman’s equation, the 
dynamic programming method is not used in the paper. 

If condition (1) is not satisfied, a system with prediction can be 
used as a convenient deterministic analogue of a stochastic system. 

It is interesting to note that if the interference is a purely random 
process, the trajectory, although differing from the trajectory in a 
system without interference, proves to be non-random (this follows 
from the theory of large numbers 1 , and was noted by N.N. Krasovskii). 
To the other questions I can reply as follows : 

(1) If in the feedback channel (or the input channel) there is a filter, 
the variation of the interference distribution function must be taken 
into account. It is easy to show that the variation ofthe spectral prop¬ 
erties of the interference does not change the algorithm, provided the 
current value of the interference is not being measured by the con¬ 


troller. Since filtration of the error signal will also take place in such a 
system, a predictor must be connected ahead of the optimal controller 2 . 

As before, the main element of such a more complex system remains 
the controller, the algorithm of which is determined in the paper. 

(2) The paper considers a system in which the /?th derivative ofthe 
output signal is constrained. 

(3) The optimal switching line for a second-order system, with 
interference 7? 0 and rj l9 constrained in amplitude, present, is shown in 
Figure A. Here a is the optimal switching line in a system without 
interference, be and de the optimal switching lines in the second and 
fourth quadrants with interference present, ABO the optimal trajectory 
without interference, and ACDEO the optimal trajectory with inter¬ 
ference (for purely random noise the trajectory is the only one). Both 
trajectories contain two intervals. 



Figure A 


(4) As a rule r 

y= I o. 

£= 1 

If, however, y < 0, eqn (30) in the paper remains valid with y re¬ 
placed by \y |, if the sign of the right-hand side of eqn (22) is changed. 
This is explained by the fact that the amount of information fed 
along the channel is determined by the modulus of quantity y. 
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R. E. Kalman, Research Institute for Advanced Studies , Centre for 
Control Theory , 7212 Bellona Avenue , Baltimore 12, M.D., U.S.A. 

Dr. Novoseltsev’s ideas are quite interesting. Since his approach is a 
rather obvious one, a much deeper investigation would be required 
before the significance of his ideas can be assessed. There are three 
ways in which this could be done: (1) Rigorous mathematical methods; 
(2) Intuitive engineering arguments, together with a precise statement 
of unproved assumptions; and (3) Intuitive engineering arguments, 
together with a careful experimental verification of the assumptions. 
The paper does not meet any of these requirements. 

The work of Bellman, upon which this paper is based, has been an 
extraordinarily stimulating influence. However, it is now generally 
recognized, especially since the contributions of Pontryagin and his 
collaborators, that the basic principles of dynamic programming are 
insufficient for practical applications and a deeper mathematical in¬ 
vestigation is required in each case. (See the work of Bryson and Kelley 
in particular.) Especially difficult is the case of stochastic systems, 
where even very basic questions (existence and stability) are still 
largely unsolved. (See, however, the work of Gikhman and Krasovskii.) 
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The present opinion in the U.S.A. is that the theory of the effect 
of random disturbances, in non-linear dynamic systems, is presently at 
too primitive a level to warrant crude empirical applications of dynamic 
programming (as was done in this paper) to actual engineering prob- 

lem in the absence of solid theoretical knowledge it would be dangerous 
to claim that practical problems can be attacked effectively by a crude 
simplification of the real problem. It is good to base engineering argu¬ 
ments on a simplified scientific theory, but simplicity in itself is no 
guarantee of validity. Our experience in the U.S.A. is that oversimpli¬ 
fied approaches, as in this paper, do not really contribute to the solu¬ 
tion of practical problems. 

V. N. Novoseltsev, in reply 

Obviously the method of dynamic programming does not permit us at 
present to obtain a solution to the problem of synthesis of an optima 


algorithm (e.g., Reference 2 in the paper). It is therefore justifiable to 
seek other methods of solution, among which is the method examined 
in the paper. This is based on replacement of the ordinary optimum 
criterion by a simpler auxiliary criterion, and does not involve the use 
of dynamic programming, although based on the same initial relation- 

thorough experimental verification and development of the results 
presuppose: 

(a) Direct simulation of the optimal control algorithms according 
to the formulae obtained. Some results of such simulation are given m 
the paper. 

(b) Static synthesis of optimal algorithms. Original work in this 
field is being done by R. I. Stakhovskii, using multi-channel automatic 
optimizers. 

( c ) Verification on prediction control systems, when the assembly 
average of the random trajectory in the future is predicted directly. 
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Quasi-invariant Hybrid Multi-parameter Control Loops 

V. STREJC 


Summary 

The article describes the general theory of the synthesis of hybrid, 
multi-parameter control systems acted upon by disturbances of an 
arbitrary form and with constant command input signals. The control 
loops contain controllers which operate in parallel and can be realiz¬ 
ed by means of an automatic digital computer and continuously acting 
controllers. A solution is presented according to which disturbances 
u k efl fect only the controlled variables x it i = k, and the degree of this 
effect can be limited in accordance with the selected criterion of the 
quality of control, or according to other suitable conditions of control. 
Control loops of this kind are called quasi-invariant control loops. 
The paper describes the application of the criterion of the finite number 
of control steps and of the criterion of the least square of deviations. 
The validity of the solution of the Wiener-Hopf integral equation is 
extended to control loops containing digital computers. 

Sommaire 

Le rapport presente la theorie generale de la synthese des systemes de 
reglage hybrides a plusieurs variables, influences par les perturbations 
d’une forme arbitrage et soumis a des grandeurs de commande cons- 
tantes. Dans les systemes de reglage sont incorpores en parallele des 
regulateurs a action discontinue, realises par un calculateur numerique et 
des regulateurs. On a montre une solution d’apres la quelle les per¬ 
turbations u jc influencent seulement les grandeurs reglees x^ i = k, 
avec la possibility de reduire le degr6 de cette influence d’apres un 
critere choisi de la qualite de reglage ou d’apres d’autres conditions 
a Ppropriees. Ces systemes de reglage sout designes comme «systemes 
quasi invariants». Dans ce travail on a applique le critere du nombre 
■ fini de pas de reglage et le critere du minimum de l’ecart quadra- 
tique moyen et on a etendu la validite de la solution de l’equation 
integrate de Wiener-Hopf a des systemes de reglage contenant des 
calculateurs numeriques. 


Zusammenfassung 

Die Arbeit beschreibt eine allgemeine Theorie zur Synthese von ge- 
mischt analog-digital (hybrid) arbeitenden Mehrfachregelkreisen bei 
beliebigen StorgroBeneinwirkungen und konstanten FiihrungsgroBen. 
Die Regelkreise enthalten parallel arbeitende diskontinuierlich (Digital- 
rechner) und kontinuierlich wirkende Regler. Eine Losung wird an- 
gegeben, bei der die StorgroBen u h nur die RegelgroBen x i9 i = k be- 
einflussen; der Grad dieserBeeinflussung laBt sich gemaB der gewahlten 
Regelgute Oder nach anderen zweckmaBigen Bedingungen der Rege- 
lung beschranken. 

a ^ e f e !!^ ei ? e d ! eser . Art werden als „quasi-invariant“ bezeichnet. Der 
Aufsatz beschreibt die Anwendung der Methode der Regelung durch 


eine endliche Anzahl von Schritten und das Kriterium der minimalen 
quadratischen Abweichung. Die Giiltigkeit der Losung der Wiener- 
Hopfschen Integralgleichung wird auf Regelkreise mit Digitalrechnern 
erweitert. 


Introduction 

Previous papers by the author 1 " 3 contain the general theory of 
the synthesis of control systems and of the compensation of the 
effects of disturbances in hybrid, multi-parameter control loops, 
with due consideration of the conditions of autonomy, invariance 
and the finite number of control steps. A control loop is regarded 
as hybrid if the function of the controller is performed by a 
discrete filter (digital correcting member), the realization of 
which is assumed to be attainable by an automatic digital com¬ 
puter and a continuously-acting controller. In practice, hybrid 
control loops can be formed by the addition of an automatic 
computer to control loops containing continuously-acting con¬ 
trollers. This arrangement is made either in cases where it is 
necessary to improve the quality of control and to attain a higher 
stage of complex automation that would be difficult or too 
costly to realize by other means, or in newly designed control 
systems with the automatic computer as the main technical 
means of realizing automation and in which the simple, con¬ 
tinuously-acting controllers are used as a stand-by for sustaining 
the operation of the control system in the case of an outage of 
the automatic computer. 

In practical applications the case of a multi-parameter control 
system may occur frequently where the desired values of the 
controlled variables remain constant (their relative deviations 



Figure 1 
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being zero), and the task of the control system is confined to the 
compensation of the effects of disturbances. If a control-system 
structure, according to Figure 1, is selected for a multi-parameter 
control loop of this kind, the conditions of invariance cannot 
be fulfilled. However, the existence of a solution will be pre¬ 
sented according to which only the controlled variables xt, i = k, 
are influenced by disturbances u k with the possibility of deter¬ 
mining the limits of this influence, according to the selected 
criterion of the quality of control, or according to other suitable 
control conditions. Let control loops of this kind be designated 
as quasi-invariant control loops. 

For a control loop according to Figure 1 : 

[K;(z, 0)]- {[1]■+ [O'(2,0)] [P* (2,0)]}- [Oj(2,0)] (1) 

[Jf;(2,8)]-[o:(z,«)]-[Q*(2,«)][P*(2,0)]K(z0)] (2) 
where 

[a GO]=([i]+[s GO] [* (pW 1 [G GO] 

\Q U GO]={[i]+[s GO] [R (pW 1 l°u GO] 

In the compensation of disturbance effects [K* u (z, s)] and 
\K* U (z, 0)] are the matrices of the transfer functions of closed 
control loops with the elements of the matrices expressed as 
discrete Laplace transforms (Z transforms). In eqn (2) s stands 
for the relative value of the independent time variable which 
during one interval of sampling attains the value of e < Oh- 1 >. 
The sampling interval T is constant, and let the sampling be 
synchronous at all points of the control loop. 

[u (f)] is the (£; 1) type column matrix of the disturbances 
[x (f)] is the 0; 1) type column matrix of the controlled variables 
[5 (p)] is the (v; p) type rectangular matrix, p > v of the transfer 
functions of the controlled system containing a servomotor 
and a final control element 

[G 00] is the (v ; «) type rectangular matrix, p > v, of the transfer 
functions of a controlled system containing a servomotor, 
final control element and a holding member 
[G u O)] is the (v ; f) type rectangular matrix, f ^ v, of the transfer 
functions of the controlled system containing a holding 
member 

[p*00] is the (p ; v) type rectangular matrix, p>v, of the 
transfer functions of digital correcting members 
[R 00] is the O; v) type rectangular matrix, p > v, of the transfer 
functions of continuously-acting controllers 


where ^|[ ffl * (z ,0)] = [fi*(z,0)]' 1 (5) 

A nB 0) 

As the continuously-acting controllers, the transfer functions 
of which have the matrix [R (p)l are determined by the condition 
of all loops of the control system remaining stable in the case 
of a computer outage, it may be stated that the elements of 
matrix [Q* u (z, e)] will always be stable. 

On the other hand, the elements of the second term on the 
right-hand side of eqn (4) can be unstable if the polynomial 
A QB ( z , 0), which is the numerator of the determinant f (z, 0) 
of matrix [£* (z, 0)], has its zero outside the zone of stability. 
The unstable zeros of polynomial A q B (z, 0) must be assumed, 
however, to be compensated by the numerator of the elements 
of matrix {[Q» (z, 0)] - [J£ (z, 0)]\ which, as can be seen 
from eqn (3), is a cofactor of the matrix [P* (z, 0)]. In accord¬ 
ance with the assumptions stated previously, the elements of 
matrices [Q* (z, e)] and [co* (z, 0)] in eqn (4)^ are stable, while 
the stability of the elements of matrix [K* (z, 0)] must be 
presupposed. On the basis of the above findings, it is possible 
to state the condition of the stability of a hybrid, multi-parameter 
control system for the compensation of disturbance effects, as 
follows: 

[12* (z, 0)] - [K* (z, 0)] = A« b (z, 0) [Dj (z, 0)] (6) 

where [D* (z, 0)] is the matrix of auxiliary functions that must 
be determined in more detail, while A QB (z, 0) follows irom 
the equation 

(z, 0)=A^ 8 (z, 0) A^ b (z> 0) (7) 


where Aq B (z, 0) signifies the product of the stable, and A ftB (z,0) 
the product of the unstable root factors of the numerator of the 
determinant of matrix [12* (z, 0)]. 

Introduce 

[k; (z, 0)] = [3„* (z, 0)] [(1 -z-y C* (z, 0)] (8) 


where [Q' u (z, 0)] is the matrix of auxiliary functions, and 
[(1 — z~ x ) m C* (z, 0)] is a diagonal matrix of the (I; I) type, the 
elements of which should be polynomials independent of the 
properties of control loop members. Let these elements be the 
denominators of the Z transforms of the general form of the 


disturbances 


[L*(z,0)] = 


F*(z,0) 

_(1 —z -1 ) m C*(z, 0)_ 


(9) 


Now, eqn (6) can be rewritten in the form 


Conditions of Stability 

As it is desirable to express the quality of control by the 
requirements upon the transfer functions in matrix [K u (z, 0)], 
the matrix [P* (z, 0)] is a function of matrix [K" u (z, 0)]. It can 
be calculated from eqn (1) that 

[P* (z, 0)] = [fl* (z, 0)]’ 1 {[f2„* (z, 0)] - [K* u (z, 0)]} 

[•K* (z, 0)] “ 1 (3) 


By substituting relation (3) for [P* (z, 0)] into eqn (2) 

[Kt (z, s)] = [o; (Z, £)] - \P* (z > £ )] [®* 

{[<2* (z, 0)] — \_K* (z, 0)]} (A 


[at (Z, 0)] - [Qt (z, 0)] [(1-z- 1 ) m C* (z, 0)] 

= Afig (z, 0) [D* (z, 0)] (10) 

After substituting relations (8) and (10) into eqn (3), matrix 
[P* (z, 0)] expressed by this equation will acquire the form 

[P*(z,0)]=^Li%co*(z,0)] [D* (z, 0)] 

A^ b (z,0) 

m 0)] [(1 - Z- T C* (z, 0)]} " 1 (11) 

t The ‘determinant’ of the (m; n) type rectangular matrix A, with 
m < n, is to be considered as being identical with the determinant of 
matrix' A T A where T A is the matrix transposed towards matrix A. 
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Now, let the real functions in eqn (2) be marked with the sub¬ 
script j, and the imaginary functions in eqn ( 11 ) with the sub¬ 
script p. After substituting relation (11) into eqn (2), it follows 

[Oz,8)] = [fi£M] 

- (z ’ e E K (z, 0)] [A* (z, 0)] (12) 

provided that 

Wup (z, 0)] " 1 IQ* (z, 0)] = [1] ( 13 ) 

Assumption (13) can be fulfilled only if the zeros and the poles 
of the determinant of matrix [Q’ u (z, 0)] are inside the zone of 
stability. 

The following holds for the elements of matrices in eqn (10): 

Q l ik (z, 0) - 5 * tk (z, 0) (1 -z- 1 )" C* (z, 0) 


r = Aqb (z, 0) D* ik (z, 0) 

Introduce 


(14) 


Let the degree of polynomial Q’ uB , (z, 0) be assumed as 

Qb = Q + N ( 2i) 

where Q is the lowest possible degree of the polynomial 
QuB , ik (z, 0) that follows from eqn (19), and N the number of 
degrees of freedom. 

Assuming that 

Qa + M b <Q b + M a 4- 777 + C (22) 

the degree of the resultant polynomial on the left-hand side 
of eqn (19) will be 

QB+M A + m + C = L 1 +D + N (23) 

From eqn (23) follows the degree of polynomial Q* u B ik ( Z) 0) 

Qb — L 1 -\-N = Q + N 
and the degree of polynomial D* uSt ik ( z , 0) 


K* ik (z, 0) ~ — g ' - — 0 ) 


AU(z,0) 


*£.u(z, 0) 

_ MuB, ik(£> 0 ) 
Ki.ik(z, 0 ) 


Q* ik ( z , 0 )= °' > 1 
QuA.iki^, 0 ) 


AU(z, 0 )= 


£>* 


uB, ik 


(z,0) 


Atr.ifcCz.O) 


(15) 


where fractions (15) represent the ratios of polynomials in z - 1 
with a finite number of terms. By using relations (15), eqn (14) 

can be rewritten in the form 


^kb, ,-fc(z, 0 ) 

(z, 0 ) 


M* 


uA, ik 


QuB, ik(z, 0 ) 

AWz, 0 ) 

D* 


(l-z-^C&frO) 


= A^( Z ,Q) %^M) 


tt(z, 0 ) 


(16) 


D = M a + m 4- C (24) 

D b = D + N (25) 

By comparing the coefficients of the equal powers z " 1 in the 
resultant polynomials on both sides of eqn (19), the system 
of Qb -F D linear algebraic equations is obtained where 

Qb + D^L^D + N (26) 

To this system of equations it is necessary to add further N + Q A 
equations of conditions that follow from the selected conditions 
of control. The system of Q B + D B -f Qa equations obtained 
m this way ^determines the coefficients of polynomials 
QuB, ijc ( z > 0 ), Qua, He (z> 0 ) and JD uB ilc (z, 0) of the auxiliary 
functions, provided that the determinant of the equation system 
oes not equal zero. The number of such coefficients is 


Similarly, the following holds for the elements of the matrices 
m eqn ( 8 ) 

ja..feo) a,.(i,o) (1 ' 7> 

In the case of 

(Z, 0) = M„\ ik (z, 0) q: A: ik (z, 0) (18) 

eqn (16) will acquire the form 

Qua, ik ( Z > 0) M* b ik (z, 0) 

- QuB, ik (z, 0) M* a< [k (z, 0) (1 -z~% C* (Z, 0) 

= 4ob(z, 0) D* Btk (z, 0) ( 19 ) 

Denote v J 


A -(*’°) =I + Z Az" v Q* uAtik ( z , 0) = 1+ f P V 1 

•-* ,-i 

CfeO) = 1+1 a „ (z , 0)= I* 


V= 1 

Ma 


v — 1 


M u a, tk (z, 0) = 1 + X a v z~ v D„\ it (z,0)= £ * 

V=1 v=x 


Mb 

J'C.ttCz, 0)= 2 /? v z" 

V=1 


( 20 ) 


2^ + Q j b + D b =:L 1 +D + 2A' + (2 1 (27) 

A more detailed analysis would prove that ft = q x and ft = 0 
holds generally, and consequently the number of conditions 
necessary for the determination of the coefficients of auxiliary 
functions may be reduced by two. 

The solution is somewhat simplified if it can be stated that 

It follows K “ A ’ tk ^ Z> ^ = ik ^ = ik °) (28) 

D U A, ik ( z > 0) = M uA) ik (z, 0) (29) 

and eqn (16) will assume the form 

MuB, ik ( z > 0) — Q ug ik (z, 0) (1 — Z 1 )™ k C* k (z, 0) 

~ ^ni3 ( z > 0) D uB> ik (z, 0) (30) 

The above simplification does not allow the inclusion, in the 
characteristic equation of transfer functions K* Ut ik (z, 0), of 
additional requirements above those asserted in the character¬ 
istic equation of the terms Q* Ui ik (z, 0 ). 

After the determination of all elements of matrix [Q* (z, 0)1 
it is necessary to check the zeros in the numerator of the deter¬ 
minant of this matrix. 

Now, the conditions of stability can be summarized as: 

Theorem 1 In the defined hybrid control loop where AFr (z 0) 
is the product of the unstable root factors of the numerator of 
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the determinant of matrix (z, 0)], with the poles of this 
determinant lying within the stable zone of plane z, the transfer 
functions of the control loops, i. e. the elements of matrix 
[Z* t (z, e)], are stable, provided that: (a) the poles and zeros 
of “the determinant of matrix [Q * (z, 0)] lie within the stable 
zone of plane z, (b) the matrix [Q‘, (z, 0)] is in accordance 
with the equation of conditions (10) and none of its elements 
is equal to zero, and (c) the poles of the elements of matrix 
[Z>* (z, 0)] in eqn (10) also lie in the stable zone of plane z. 
These conditions are necessary and sufficient. 

The Conditions of Zero Offset 

Provided that the conditions of stability are fulfilled, it is 
possible to state the condition of zero offset according to the 
theorem of finite values by the following equation: 

lim K* ik (z, 0)=0 (31) 

z-»l 

The above condition can be fulfilled if the value of wi in the 
general relation (8) is at least m = 1. In other words, the product 
of the numerator root factors of transfer functions K u s n c ( z , 0) 
must necessarily contain the factor (1 — z" 1 ). 


Finite Number of Control Steps 

In the compensation of disturbance effects a multi-parameter 
control loop complies with the requirement of a finite number 
of control steps, if the same requirement is complied with by 
all components of output signals X\(z, e) of the controlled 
system. In this case 

X* k (z,t.) — K*' it(z,s) U* (z,0) (35) 

The finite number of control steps is understood as the number 
of sampling intervals, at the attainment of which the offset is 
permanently zero or constant at any one instant of sampling. 
In the intervals between the instants of sampling this condition 
need not be fulfilled. If it is possible to express the Z transforms 
of the general forms of the disturbances by eqn (9), and the 
matrix of transfer functions [K* u (z, 0)] by eqn (8), it follows 

X*(z j 0)] = [S:(2 ? 0)]F ;{s (z,0)] (36) 

It follows from eqn (36) that the requirement of the finite number 
of control steps can be complied with only if the elements of 
matrix [2*(z,0)] are polynomials having a finite number of 
terms. 

In this case it is necessary to substitute in eqns (18) and (19) 


Quasi-invariant Control Loops 

Provided that all zeros of the determinant of matrix 
[fl* (z, 0)] lie within the stable zone of plane z, the following 
substitution can be made in eqn (14): 

Afl B (z,0) = l (32) 

If the selectable functions are stated as 

g„%(z, 0)=0 for i + k (33) 

Dt ik (z,0) = Qt ik (z,0) for i*k (34) 


it follows 


2i.»(*,o)-i 

Then for individual components 

xiUz,o)=e: it (z,o)F*(z,o) 
and the degree of polynomial X* k (z,0) is 
X = L 1 + N + F 

The transform of the controlled variable is 


and the remaining functions D" Ut ik (z, 0), i = k, can be deter¬ 
mined by the same method as shown earlier in this paper. In this 
case the matrix [Q’ u (z, 0)] will be a diagonal matrix and con¬ 
sequently, with regard to eqn (8), [K u (z, 0)] will also be a 
diagonal matrix. This solution permits a situation to be leached 
where disturbances U* c (z, 0), (k = 1,2,..., f), (where ^ > v 
and v is the number of controlled variables) will influence only 
the controlled variables X‘ (z, e), for which i = k, and will 
have no influence upon the controlled variables X ) (z, s ), for 
which i t* k. If £ < v, the effect of disturbances U k (z, 0) will 
be confined to the controlled variables X■ (z, s), for which 
/ = k and / = 1,2 ,...,? and with no effect upon the controlled 
variables X■ (z,«), for which i ^ k and also those for which 

i — k but i = I + 1, f + 2, ..., v. ■ 

Due to this solution the transfer functions in diagonal 
matrix [K* u (z, 0)] can have an arbitrary number of degrees of 
freedom that can be utilized for the fulfilment of further con¬ 
ditions of control, or for the compliance with a suitable criterion 
of the quality of control. In this way it is possible to reach a 
solution at which the effect of disturbances, that cannot be 
eliminated by the introduction of condition (33), is kept within 
admissible limits. 

In principle, this method of the compensation of disturbance 
effects can also be applied to continuously-acting control 
systems. However, up to the time of writing this paper, this 
possibility has not been mentioned in any technical literature 
accessible to the author. 


Xf(z, 0)= E xf k (z, 0) 


(37) 


(38) 


(39) 


(40) 


/C=l 


with the degree of polynomial X t * (z,0) being 

X- = L 1 + {N + F) i (41) 

where (N + F) t is the highest value of the sum N + F in the 
polynomials X* k (z,0), (k — 1, 2,..., I). 

The number of the control steps is thus 


n ki =L 1 +(N+F) i +l 


(42) 


The highest value of n u (/ = 1, 2, ..., v), is regarded to be the 
finite number of the control steps of the whole multi-parameter 

If disturbances u k (t) can be regarded as the linear combi¬ 
nation of the function r m_1 /(»i —1)!, it follows F = m— 1 and 
the number of control steps is 


na-Li+CAf + m), 


(43) 


It can equally be proved that it is also possible to obtain a 
zero deviation of the controlled variables X- (z, e) for e < 0 4-1 > 
beginning with the instant n = n ki provided that: disturbance 
u k {t) varied from instant n = 0 according to the function u k (t) 
J r m-i j( m _ i)! ; the elements of matrices [G(p)] and [G u {p)] 
have at least one ra-fold zero pole (in hybrid loops the elements 
of matrices [G(p)] and [G„(p)] must have a holding member at 
least of the order (m - 1)), in the equation of conditions (6) and 
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in_the equations derived from it A nB (z,0) is substituted for 
&QB (z>0) and the auxiliary functions in matrices [D* u (z, 0)] 
and [Ql (z,0)] are only polynomials in z~\ 

Then it follows from eqn (16) 

M*„, ik (z, 0) 

- Sub, ik (z, o) (i 1 c* (z, o) m; A: ik (z, 0 ) 


= A ob (z, 0) ik (Z, 0) M* a ' ik (z, 0) 

(44) 

The degree of eqn (44) is 

Q B + m-i + C + M A = l 1 +D + N+M A 

(45) 

d b =d+n 

(46) 

D = m-1+C + M A 

(47) 

Q B = h + N + M A 

(48) 

The number of control steps is then 

n ki — (Qb + F)i 

(49) 

n ki~h 4-(M^ + iV+C + mX-—1 

(50) 

where 4 is the degree of the polynomial A ns (z, 0). 

Let it be noted further that in eqn (44) 


[r*(z, s)] = the (v; £) type rectangular matrix of the ideal trans¬ 
fer functions of the control system 
[A (t)] = the deviations of the controlled variables x [(/)] f rom 

the ideal output signals \y(t)]. 

A ■ i (n,s)=x i (n,e)-y i (n,s) (53) 

For the sake of brevity the analysis that follows deals only 
with the case of non-correlated input signals. By using the 
results published in an earlier paper by the author, the transfer 
functions sought for, i.e. the elements of matrix K‘ ( z,e )], can 
be determined by the solution of equation 

K-u,ik (jco, s) S kk (co, 0) — rf k (jco,s') 2 S* k (co, 0) = 0 (54) 

where K* Uktk (Jco, e) and r* k (jco, e) are the z transforms of the 
above-mentioned transfer functions, z = e-'", co = a>T while 
1 Skk (.co, 0) and 2 S kk (co,0) are discrete forms of the performance 
spectral densities: 

’ S *c (co, 0) = S* kUk (co, 0) + S* k „ k (co, 0) (55) 

2 S* k (co, 0) = S* kUk (co, 0) ( 56 ) 

Eqn (54) is representing the discrete Laplace transform of the 
Wiener-Hopf integral equation, the solution of which 


D: B , ik (z, 0)= x d vZ - 


v = 0 


(51) 


K u, ik (Jco, e) = 


J7 k (jw,s)y k (co,0 ) 
'kk ( CO, 0) 


l S7‘ 


differently from the polynomial in eqn ( 20 ). Owing to d 0 * 0 X S kk (co, 0) 

it has been possible to reduce exponent m in eqn (44) It should i ^ 

also be mentioned that in this case the number of control steps b * kn ° Wn ^ h ° d fulfUs th ® condition 
cannot generally be lowered by the value of M A by setting j Cu , fc ( n> £ ) = q f()r n<Q 


(57) 


n* ^ m_ D'ub, ik (z, 0) 

U u, ik o) — ~rz -—— 

m „a, ik (z, 0) 


(58) 


because the output signal of the digital correction members is 


where k u<ik (n,s) is the original of the transform Kl n -(iw s) 
In eqn (57) , l U ’ 

' S kk (co, 0) X (©, 0) = (m, 0) 


(59) 


E* 2 (z, 0) = - A qa (z, 0) [co* (z, 0)] [D* (z, 0)] U* (z, 0)] (52) 

and it cannot be assumed that in a general case , n . (z, 0 ) is 
contained in A qa (z,0). The denominators of the elements of 
matrix [co* (z, 0 )] are contained in A flvl (z, 0 ). 


whereall the poles of 1 S kk (oj, 0) are inside, and all the poles of 
S kk (co, 0) are outside the zone of stability of plane z. The + sign 
in the place of a subscript of the brackets in the numerator of 
eqn (57) signifies that the function in the brackets has all its 
poles inside the stability zone of plane z. 


The Optimum Compensation of Disturbance Effects in Wiener’s 
Sense 

A method has been shown for limiting the effect of dis¬ 
turbances in quasi-variant control loops by the criterion of the 
finite number of control steps being considered as the criterion 
of the quality of control. Another method of solution will be 
shown where the least square of the deviations of the controlled 
variables is taken as the criterion of the quality of control. Let 
the problem be stated by the application of the conventional 
diagram shown in Figure 2 with the following meaning of de¬ 
notations : 

[«(/)] = stationary random disturbances 

[m(t)} = parasitic noise 

[A*(z,c)] = the (v; £) type rectangular matrix of the transfer 
functions of the control system that are to be deter¬ 
mined 



Figure 2 


For s == 0 the elements of matrix [<(z, 0)] can be determined 
trom eqn (57), and subsequently the matrix of the transfer func¬ 
tions of the digital correcting members is determined from 
eqn (3). 

Equation ( 2 ) represents the unequivocal relationship that 
exists between transfer functions K* ik (z,0) and K* u ik (z,s), 
with the former necessarily fulfilling eqn ( 6 ) and also the con¬ 
ditions of stability attached to this equation. In order to ensure 
the stability of transfer functions K l,,j(z,e), it is necessary 
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that the conditions of stability related to eqn (6) are also ful¬ 
filled by transfer functions K*jj(z, 0). This requirement can be 
complied with in various ways, one of which is stated below. 

If, on the other hand, transfer functions K^ij(z, 0) were not 
fulfilling the stability conditions related to eqn (6), the required 
course of the controlling actions would be ensured only at the 
instants of sampling by the digital correcting members calculated 
from eqn (3) and with the aid of transfer functions K* Jk (z, 0) 
determined by eqn (57). However, during the periods between 
the sampling instants, the course in time of the controlled vari¬ 
ables cannot be guaranteed, and it may even be labile. 

The new concepts can be summarized in the following 
theorem. 

Theorem 2 —For the determination of the digital correcting 
members in Wiener’s sense, i.e. in a control loop containing a 
continuously-acting controlling system and exposed to the effects 
of stationary random disturbances, the mean square of the de¬ 
viations of the real output signals from the ideal output signals 
should attain its minimum value, the command transfer func¬ 
tions Kujk (z,0) must fulfil the conditions of stability issuing 
from the solution of the Wiener-Hopf integral equation, and, 
in order to ensure the stability of transfer functions K U) i k (z,s), 
the transfer functions /& (z, 0) of this solution must comply 
with the conditions of stability pertaining to eqn (6). 

These conditions are necessary and sufficient. This is the 


fundamental difference between the described control loops and 
control loops containing only discretely-acting or only con¬ 
tinuously-acting members. 

The conditions stipulated in Theorem 2 can be fulfilled, if 
the root factors in the numerators and denominators ofttansfer 
functions Ku,ijc (z,0) derived from the relations 0) and 

2 S& (6>>0) are introduced as a condition into auxiliary functions 
Q* ik (z, 0) and D* lAlt (z, 0) calculated from the equation of con¬ 
ditions (6). From this point of view, the solution according to the 
least square of deviations in Wiener’s sense represents only the 
utilization of a possible application of the required criterion of 
quality of control within the determinative synthesis theory, and 
the possibility of extending the auxiliary functions 
and Dl ik (z, 0) by the required number of degrees of freedom. 
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Optimum Control of Discrete-time Dynamic Processes 

B. FRIEDLAND 


Summary 

The first part of the paper deals with the general problem of forcing a 
non-linear process to follow a specified (discrete-time) trajectory. The 
performance is measured by a functional of the vector error and input 
sequences and the inputs are assumed to be constrained. By the use 
of standard methods of calculus the optimum controller is shown to 
require the solution of a system of difference equations of twice the 
order of the process to be controlled with boundary conditions at the 
terminal instant dependent on the performance measure. A simple 
geometric interpretation of the problem is given. 

The second part of the paper is concerned with a more specific 
class of problem in which the process is linear, and the performance 
objective is one of the following: 

(1) To force the process from an arbitrary initial state as close as pos¬ 
sible to a specified terminal state in a fixed number of steps. 

(2) To force the process from the initial state to a specified terminal 
state in a fixed number of steps with minimum ‘effort 5 . 

(3) To force the process from the initial state to a specified terminal 
state in a minimum number of steps. 

A special-purpose computer which performs the required compu¬ 
tation in real time is proposed as the on-line optimum controller. 


Sommaire 

La premiere partie de la communication traite du piobleme general 
consistent a obliger un processus non-lineaire a suivre une trajectoire 
specifiee en temps discret. La performance est mesuree a l’aide d’une 
fonctionnelle de l’erreur de vecteur et des sequences d’entree et les 
signaux d’entree sont presumes limites. II est montre, a l’aide de 
methodes de calcul habituelles, que le regulateur optimal exige la 
resolution d’un systeme d’equations aux differences finies dont l’ordre 
est double de celui du processus devant etre regie, avec des conditions 
aux limites a Tinstant final dependant de la mesure de la performance 
Une interpretation geometrique simple du probleme est donnee. 

La seconde partie de la communication se rapporte a une categorie 
plus specifique de problemes, dans laquelle le processus est lineaire et 
Tobjectif de la performance est Tun des suivants: 

(1) Amener le processus d’un etat initial arbitraire au voisinage le 
plus proche possible d’un etat final determine en un nombre d’etapes 
donne. 

(2) Amener le processus de l’etat initial a un etat final determine en 
un nombres d’etapes donne avec l’«effort» minimal. 

(3) Amener le processus de l’etat initial a un etat final determine en 
un nombre d’etapes minimal. Un calculates a usage special qui 
effectue le calcul necessaire en temps reel est propose en tant que 
regulateur optimal en ligne. 


Zusammenfassung 

Der erste Teil der Arbeit befafit sich mit dem allgemeinen Problem 

r" n “ hB, T P , r ° ZeB 211 zwin S en > einer festgelegten Trajektorie 
(fiir diskrete Zettpunkte) zu folgen. Der Verlauf wird auf Grund des 

K deS -T ekt0rS ^ de “ FeWer Und der Eingangsfolgen ge- 
d ^ bei lst • angenommen, daB die Eingangsgrofien beschrankt 

RelLei^T^ T 81 Berechnun gsmethoden fur optimale 
vnn^f^’-ru Lf ng eines Systems von Dififerenzengleichungen 
von der zweifachen Ordnung des zu regelnden Prozesses erforderlich 


ist; die Grenzbedingungen im Endzustand sind dabei von dem ge- 
messenen Verlauf abhangig. Eine einfache geometrische Deutung des 
Problems wird gegeben. 

Der zweite Teil der Arbeit beschaftigt sich mehr mit einer speziellen 
Klasse von Problemen, in denen fur einen linearen ProzeB einer der 
folgenden Gesichtspunkte zugrunde liegt: 

Der ProzeB wird gezwungen: 

1. von einem willkiirlichen Anfangszustand aus so genau wie moglich 
einen festgelegten Endzustand in einer bestimmten Anzahl von 
Schritten zu erreichen, 

2. von dem Anfangszustand zu einem festgelegten Endzustand in 
einer bestimmten Anzahl von Schritten mit minimalem Aufwand 
zu kommen und 

3. von dem Anfangszustand zu einem festgelegten Endzustand in einer 
minimalen Anzahl von Schritten zu gelangen. 

Ein spezieller Rechner, der die notwendigen Bercchnungen in 
Echtzeit ausfiihrt, wird als ein direkt arbeitender (on-line) optimaler 
Regler vorgeschlagen. 


Introduction 

Traditionally, feedback control systems have been designed by 
the use of techniques in which the configuration of the controller 
is assumed and cut-and-try procedures are used to determine the 
settings of one or more adjustable parameters, such as gains, lag 
or lead times, within the controller. These techniques have 
proved to be inadequate for many problems in which the proc¬ 
ess to be controlled is non-linear, in which the actuating forces 
are constrained, and where high performance is essential. Con¬ 
sequently there has been an increasing interest in the develop¬ 
ment of a systematic theory of optimum control which does not 
suffer from these shortcomings. Numerous investigations carried 
out during the past decade with the aid of such mathematical 
tools as variational calculus (including its modification by Pon- 
tryagin and his students), functional analysis, and dynamic pro¬ 
gramming have uncovered results of considerable promise, but 
which have revealed computational difficulties which tax the 
abilities of even the most advanced electronic computers. 

The present paper has a twofold objective: first, to show a 
representative approach to the problem of optimum control 
system design and the computational problem to which such an 
approach leads; and second, to display a special-purpose com¬ 
puter which could be used to realize the optimum controller for 
a limited class of control problems. 

Formulation of the Problem 

Consideration here is limited to discrete-time processes 
whose behaviour can be represented by a vector difference 
equation 

x (n +1) = p [n, x (ft), u (ft)] ft = v, v +1,..A — 1 (1) 
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where x{n) is a k vector called the state of the process at the nth 
step and u (n) is an /* vector called the input. The control is to 
be initiated at n v at which time the process is in state x(v). 
The objective of the design is to determine an algorithm for 
generating a control sequence u(v\ u (v + 1), u(N — 1) 
which causes the sequence of process states x(y), x(y-{- 1), ..., 
x (N) to deviate as little as possible from a specified sequence 
y (p), y(*' T 0, • yW, but not at the expense of using exces¬ 
sively large inputs. To accomplish this objective a performance 
measure is defined 

F 0 = F 0 (e(v) . e(N);u (v), ...,u(N- 1)) (2) 

where 

e(n) = x(n)-y(n) (3) 

which the optimum input sequence is to minimize. It may also 
be required that the process terminate at a specified state or 
region and this is done by imposing the condition 

L(e(N)) — 0 (4) 

Moreover, the choice ol' inputs is restricted since they may 
be physically limited in amplitude, total energy, or in some 
other way. These limitations arc expressed in the form of s con- 


take the process to the optimum next state, at which time the 
computation will be repeated. It is thus clear that the algorithm 
by which the first input in the optimum sequence is calculated is 
the realization of the optimum feedback control law, while the 
algorithm for calculating the entire input sequence is a method 
of determining the optimum open-loop (preprogrammed) 
control. 

Derivation of Control Equations 

The minimization problem formulated above is treated by 
the method of Lagrange multipliers. As is well known this 
method is capable of yielding only stationary points, not absolute 
minima. To avoid further difficulties it is thus assumed that the 
required minimum is actually attained at one of the stationary 
points, and is detected by a separate test if the minimum is not 
unique. 

For the use of this technique it is convenient to write the 
process equations in the following form 

e(n + l) + y(n + l)-p(e(n)-y (n),u(n),n)= 0 

n = v,..., N — 1 (6) 


straints 

F ; (u(v),..., u(IV — l))<c'i(v) 1 = 1,2,...,® ( 5 ) 

The mathematical problem which must be solved is then to 
minimize the performance measure eqn (2), which is implicitly 
dependent only on the initial state and the input sequence, since 
all subsequent states depend on these, subject to the terminal 
condition eqn (4) and the s constraints of eqn (5). Geometrically, 
the set of input sequences which result in a specified value ol F 0 
can be regarded as points on a surface in an (N - v)r dimen¬ 
sional space and the set of inputs which simultaneously satisfy 
all the constraints of cqns (4) and (5) as a region Rm the space. 
The optimum input sequence either lies within R, in which case 
the constraints can be ignored, and the minimum of F 0 occurs 
at a point where the gradient of F 0 with respect to the mpu 
sequence is zero (provided the gradient exists at that pomt), « 
else it occurs on the boundary of R, in which case the boundary 
is tangent to the surface of minimum F 0 . This tangency con¬ 
dition requires that the gradient of F 0 be parallel and m opposite 
direction to the gradient of the boundary surface. 

Before proceeding with the mathematical express on of these 

conditions, the nature of the solution will be considered - 
dcntlv the answer to the geometric problem is a sequence 
u (v) _ U (N- 1) which would be optimum with respect to the 

! ’' ’ i \ i rorresDonding optimum trajectory 

starting state x(v), and a corresponding f 

nlonc which the process would move from x(y) to x{ ) 

® * i* t chrntiH n situation arise, however, 

this sequence is applied. Should a ’ th 

where as a result of a disturbance or 

trajectory deviates front the best that can be done 
sequence will no longer be optimum. problem 

in this case is to regard as the starting 

the state that the process is then m and T allow 

optimum input sequence and correspontogWe®^ 
for the possibility that such errors may reached during t he 
control interval, it is agreed that every nro blem and a 

control interval is the starting state ° r * each P step . If this is 
new optimum sequence is recomp h entire input 

done, however, it is not necessary to computettiej ^ 

sequence, but only the first member thereof, since this p 


with the initial condition e(v) — x(y) x k(y) = 0 which is in¬ 
dependent of the control applied. It is also convenient to write 
the constraints in the following form: 

Fi-h t =0 i=l,2, ...,s (7) 

where 

hi^Ciiy) w 

Then the following functional is formed 

H=H [e (v),..., e (N); u (v),..., u (N -1)] 

= F 0 + £ LF ; + E (e(n + l) + y(n + l) 

j=l n=v 

— p(e(n)—y (n), u (n), n)'z (n) (9) 

where z(v) z(N- 1) is a sequence of undetermined Lagrange 

multipUer vectors, i, are addi.ionai Lagraog. muldpben, and 
Z prime denotes transposition, A condition for the e»,s,en» 
of a stationary point of F, subject to the consWnhs « 

rtotives of H with “'/f ZUt ““Verfonnih" 

the required partial differenti’ations with respect to the «,<«) 
the following set of vector equations is obtained 

£ A£ _^i-=P'(n) Z (n) n = v,...,N-l (10) 

i=o 5u (”) 

, w p the gradient of F { with respect to u(n), P„(«) 

is the Jacobian mateix vfifii ^® a P^ transposition" 

™d g C= T (NoteThat the gradients of the 

measure is fixed at unity, while the 2 s corresponding 
straints are undetermined.) 
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Performing the partial differentiations of eqn (9) with respect 
to the ej(ri) yields the following set of difference equations for 
the vectors z(ri) 


N - 1 


z fi(u(n)) i = 1.i 


(13) 


z(n-L) = P' x ( n )z(n)--^ri n = v,...,N -2 (11) 

where P x (ri) is the Jacobian matrix of the vector p with respect 
to x(rt), and bFJb e(ri) is the gradient of F 0 with respect to e(n). 
If e(N) may be varied to obtain the optimum input sequence, 
then the last of the sequence of vectors z(ri) is given by 


z (N — i) = 


dF 0 

de(N) 


( 12 ) 


otherwise z(N — 1) is undetermined, and eqn (12) is replaced 
by eqn (4). The system of eqns (10) and (11) are analogous to the 
Euler-Lagrange equations for the optimum control of a con¬ 
tinuous-time process which would be obtained through the use 
of the calculus of variations 1 . Equations similar to these for 
discrete-time processes were obtained by Kipiniak 2 . 

In principle, eqn (10) can be solved for the sequence of inputs 
ii (r), .. u(N 1) in terms of the sequence of adjoint vectors 
z 0')»-*-> z(N 1). Substituting the relations thus obtained into 
the original process eqns (1) and the adjoint eqns (11) which in 
general depend on the inputs through P x would result in a system 
of 2 k difference equations with s undetermined parameters 
2 1 , ..., X s . These, together with the initial conditions x(v) and the 
final condition eqn (4) or (12) and the constraints eqn (5) can be 
solved for the optimum adjoint vectors z(n ) and sequence of 
states x(n), as well as for the parameters A*. From these the 
optimum input sequence can be obtained, again by the use of 
eqn (10). (Note that if any constraint is satisfied with a strict 
inequality, the corresponding X is equal to zero.) 

If the number of steps M = N~ v is appreciable the task 
of performing the computations indicated above is formidable 
and could easily surpass the capabilities of any available elec¬ 
tronic computer. Recently several relatively effective successive 
approximations techniques have been studied 3 ' 5 but they appear 
to be useful only for precomputing an optimum trajectory for a 
single initial state and are much too slow to be used as a feed¬ 
back algorithm in real time. The next section deals with the 
realization of a special-purpose computer which, for a restricted 
class of problems, can effectively be used to perform this com¬ 
putation. 


A Special-purpose Computer for Terminal Control of Linear 
Processes 

Here the construction is considered of a computer which can 
be used to control a linear, time-invariant process to achieve 
optimal terminal performance. The following three related prob¬ 
lems are of particular concern 

(1) To force the process from the initial state to a terminal 
state which is as close as possible to a specified state in a specified 
number of steps. 

(2) To force the process from the initial state to a specified 
terminal state with minimum ‘effort’. 

(3) To force the process from the initial state to a specified 
terminal state in a minimum number of steps. 

In each case the inputs which are to be used must satisfy a 
set of constraints in the form of eqn (5) which it will be further 
assumed can be expressed in the form 


where p t is an integer ^ 0. If the ith constraint is satisfied with 
the equals sign then = c. L (v), and 


5Fj df 

du(n ) pi du(n) 


(14) 


On the other hand, if the z'th constraint is satisfied by a strict 
inequality, then the corresponding = 0. Hence eqn (10) can 
be written 


du(n) ,ti pi 


df 

du(n j 


= P' u (n)z(n) 


U5) 


For the second problem, it is assumed that the effort to be 
expended is one of the quantities constrained in the other two 
problems. In this case it is desired to minimize one of the F h 
say F l9 and eqn (15) becomes 


s 


I 
'= 1 


Pi du (ri) 


= P' u (n)z(n) 


(16) 


where X 1 = 1 and h x is the minimum effort expended—which’ is 
as yet undetermined. For the first and third problems the per¬ 
formance measure is independent of the input sequence and 
thus b FJbu (it) = 0. Hence eqn (16) can be used to express 
the relation between z (ri) and u (ri) for all three problems. 
Since b/Jb u (n ) are known (generally non-linear) functions of 
u ( n ), eqn (16) expresses an instantaneous non-linear transforma¬ 
tion from u (n ) to P u f (ri) z (ri). This transformation is seen to 
depend on and is linear in the ) H , Assuming that this transforma¬ 
tion is invertible 

u(n)=o(P' u (n)z(n);X 1 ,...,l s ) (17) 

where 6 is the inverse of the transformation expressed by 
eqn (16). Thus it is seen that under the assumptions made the 
optimum input vector at the nth instant is a known transforma¬ 
tion of the vector P u ' (ri) z (ri) dependent on s parameters! 
2i,/l 2 , which are to be determined from the constraints. 

In particular, the optimum initial input, which is the only one 
which needs to be calculated in the feedback implementation is 


u(v)=a(P' u (v)z(v);l u X 2 ,...,X s ) (18) 

The computation which must be performed is the calculation 
of the single vector z (v ) and the parameters X 1 , A 2 , ..., X s . 

In each problem the path followed from the initial state to 
the terminal state is of no concern; consequently the performance 
measure [eqn (2)] is independent of errors along the path and 
the term b F 0 /b e (ri) in eqn (11) is zero for n = v 9 ..., N—2. 
Moreover, as a consequence of the linearity and time-invariance 
assumed of the process P x = A, a constant matrix. Thus 
eqn (11) becomes 

z(n-l) = A f z(ri) (19) 

and the entire sequence of adjoint vectors can be immediately 
expressed in terms of the initial vector. (In a physical process A 
is necessarily non-singular, so the negative powers of A exist.) 
Specifically 

z(n)=A' v ~"z(v) n = v,N— 1 (20) 
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Moreover, P u (n) = B , another constant matrix in a linear, 
time-invariant system. Thus eqn (17) can be written 

u (n) = <y(B'A' v ~" z(v); A l5 ..., A s ) (21) 

For any input sequence the state of the process at the tei minal 
instant is given by 

* (tf) = A*'- v * (v) + Y A N ~ 1 ~" Bu(n) (22) 


The terminal error which would result for a given setting o 
the Xi appears explicitly to the left of the <j) unit and can thus be 
brought into the subsidiary computer for the determination ot 
the A,. It is also necessary to know whether a given setting ot 
the A- will satisfy the constraints on the u in). A realization of the 
g unit which places the entire input sequence in evidence is 
shown in Figure 2. The unit consists of a variable number of 
sections and a switching arrangement (not shown) for connecting 


Substituting eqn (21) into (22) and rearranging terms gives 
e(n) — A N v £e(lV, v) + A 1 ^;v_ v (z(v), A l9 A s )3 (23) 


where 


e(N,v) = e(v) + y(v)-A v N y(n ) 


(24) 


can be called the ‘effective error vector’ (so called because it 
equals the observed error when the desired trajectory is a 
solution to the unforced equations; if the observed error e (y) 
is used in place of e (y), this is equivalent to extrapolating the 
desired trajectory by a solution of the unforced equations), 
and the vector 

s „(z(v);A 1 ,...,A s )=' , y A- m Ba(B'A'- m z(vy,X 1 ,...,X s )(25 ) 

m = 0 

is a calculable non-linear vector-valued function of z ( v ), N-v, 

and Aj- (i = 1, 2, s). . 

The relationship between the error e (v ), z (y), and e (TV) is 
thus completely expressed by eqn (23). To obtain the relationship 
between e (v) and z (y), which determines the input to be applied 
at n = v, it is necessary to eliminate e (N) from eqn (23) and to 
evaluate the X { from the constraint relations of eqn (5). 

For the first problem, it is assumed that the terminal error 
is measured by a positive-definite function L (e (N)) which is 
thus the performance measure F 0 . Hence, from eqn (12) 


z(N — \) = A 


i v-N+l 


: (y)=-<f>(e(N)) 


(26) 


where <j> (e (AO) = 5 Lfb e (N). Thus, from eqns (26) and (23) 

_ A' N ~ v " 1 <l> [A N ~'~ 1 (Ae + g N - v )] = % (v) (27) 

where it is understood that g N - v = 9N-v (z ('■’); K • • •> D- 
solution of eqn (27) and the determination of the A, from the 
constraints gives the optimum z (v), which, in accordance wi 
eqn (18), is transformed to the optimum input u (v). This 
process is repeated at the next step with N replaced by N 1. 

A special-purpose analogue computer for performing the 
computation required by eqn (27) is shown in Figure 1 , m which 
each box represents a linear or non-lmear instantaneous trans¬ 
formation of a vector of signals into another vector^ In particular 
A is a constant linear transformation, A ” and A are 

time-varying linear transformations, j> and <7 are constant 
non-linear transformations determined by eqns (26) and (18), 
respectively, and g is a time-varying non-lmear transfoimation 
determined by eqn (25). The g unit has parameters h which must 
be set in accordance with the constraints. Since these parameters 
completely determine the g unit and the r unit, and the other 
units are determined, the only computation remaining is the 
determination of the correct settings of A,. 

The computer is operated by setting the number of steps to 
so M — N-v into the time-varying units and adjusting the A 2 - 
such that the constraints are satisfied and L (e (AO) is minimize . 



Figure 1. Structure of optimum controller 



Figure 2. Structure of a unit 


as many sections as there are steps to go. It is seen that the 
inputs which would be generated by a given setting of the A* 
appear explicitly at the outputs of a units. (Note also that the 
optimum present input can be taken from the first o unit, 
eliminating the need for the a unit of Figure 1 .) 

A useful class of constraints is represented by the general form 

F?‘ = Y 5>, (n)\ p ‘=hr l ( 2g ) 

n = v j = 1 

which is essentially a norm of the sequence u 00, ..., u {N — 1) 
in an (N - v) r dimensional space. When Pi = 2 the constraint 
is on the total available energy. When p f == 1, eqn (28) becomes 

F,= I £ K(«)l 

n - v j = 1 
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which is a constraint on the total effort, and is useful in connec¬ 
tion with control of space vehicles with limited thrust capability. 
As Pi~+ oo, Fi -> sup n>j | Uj (n) \, the familiar amplitude constraint. 
The partial derivative of F t with respect to u s (n) is given by 


where 


3F, 

uj(n) 

diij ( n ) 

hi 

| 

4-1 u 

sgn u — j 

0 u 

[ 

— 1 u 


Pi~i 


sgn Uj ( n ) 


(29) 


Thus each component of d FJb u (n) is a non-linear transforma¬ 
tion of the corresponding component of u (n) as given by 
eqn (29). 

If all the constraints are of this form, then the y'th component 
of the vector B' z (n) in eqn (16) can be written 


be assumed to exist together with either a total effort or an 
amplitude constraint.) In this case 2 3 is automatically taken into 
account, and the adjustments required are in the width of the 
dead zone and the slope of the linear region. Note that as the 
slope of the linear region becomes infinite (i.e., the energy 
constraint becomes negligible) then every input u 3 (n) is either 
zero or the maximum value h z . 

If there is only a single constraint in the form of eqn (28) to 
be accounted for, then the adjustment of the corresponding 
single parameter 2 is considerably simplified. It is observed that 
if the dot product of the vector z with the vector g is taken, 
one obtains 

N — l-v 

*'(v)tfw-,(z(v))= E z{v)A-"Ba(B’A'- n z{v)) (31) 

77 = 0 

but a (. B' A'~ n z (v )) = u (v — n). Consequently, eqn (31) can 
be written 


[£'z («)],.= E 
1=1 




M») 

h 


Pi-l 

sgn Uj (ri) 


7 = 1 , 2 , 


k (30) 


JV— 1 — v 


z '(v)g N -v(z(v))= X] G 1 («(v-n))"n(v-n) 

77=0 


Thus, for 2 ?; > 0, and at least one 2 t 4= 0, [B' z (n)] 3 is a non¬ 
decreasing symmetric function of Uj (n). Thus it is evident that 
Uj (n) is likewise a non-decreasing, symmetric function of 
[B r z («)]y. As an illustration, select p x = 1, p 2 = 2, p z = oo. Then 
the graph of [B r z (n)]j vs. Uj has the appearance of Figure 3 (b) 
and Uj vs. [ B' z (ri)]j has the appearance of Figure 3 (c). The 
non-linear characteristic o is seen to have three regions: a 
dead zone resulting from the total effort constraint, a linear 
region resulting from the quadratic constraint, and a saturation 
region resulting from the amplitude constraint. (As shown in 
Figure 3 (c) the non-linear characteristic is single valued and no 
difficulty would result in using this characteristic in the analogue 
computer. If the quadratic constraint were absent, however, there 
would be discontinuities at ± X x and we could expect difficulties 
to arise. To avoid such difficulties it may be permissible to 
assume a small but finite linear region, even when there is no 
quadratic constraint, since a quadratic constraint may always 



Figure 3. Determination of non-linear characteristic for a unit 


= E •'(»)» 1 («(«))= E E Uj{n)a~ l (Uj(n )) 

7I = V 71= V j- 1 

But t> -1 (uj (ri)) = A | iij (n)/h p ~ 1 1 sgn u } . Whence 

z'(v)^_ v (z(v))=lY E \u j \p!h p - 1 =Xh 

n = v j - 1 

Thus the value of h which is obtained for a trial value of A is 
given by the dot product of z with g divided by A: 


, T _ yO) g w - v (z(y)) 


(32) 


It is thus necessary only to evaluate this dot product to determine 
whether the constraint is satisfied. To avoid the necessity of 
adjusting the a unit in accordance with the undetermined h, 
A can be replaced by a parameter k = V,h v ~ x (p < oo). In this 
case the value of h obtained for a given setting of k is 


h = 


z '( v )gw-y(z(v)) 

k 


i Ip 


(33) 


For a single constraint the adjustment of k can be accom¬ 
plished by setting k to a very large value (which is seen to make 
the transmission through the g unit and likewise g' z small) and 
sweeping k towards a very small value (thereby increasing the 
transmission through g) until the right-hand side of eqn (33) 
is exactly equal to the available quantity c (y). Thus the sub¬ 
sidiary computer needs only to evaluate h from eqn (33) and 
compare this value with c (y) to obtain the correct setting of k. 
This technique could be used as an approximate realization for 
an amplitude constraint by making p a large but finite value 6 . 

For multiple constraints the same adjustment procedure can 
be followed, except that each of the k t = AJh/’i- 1 have to be 
adjusted in sequences of small steps and the calculation of h t 
cannot be simplified to the evaluation of a dot product, unless 
each constraint applies to components of the input vector which 
do not appear in the other constraints, in which case the dot 
product technique remains applicable. See Friedland 7 , for 
example. 
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Consider now the second problem, that of minimizing the 
total effort required to reduce the terminal error to zero (assum¬ 
ing this can be done) in a specified number of steps to go. In this 
case it is required that e ( N ) in eqn (23) be zero, and this estab¬ 
lishes the following relationship between e and z 

- A e (N, v) = g N _ v (z (v), A t ,..., K) ( 34 ) 

To solve this equation for z (v) entails the inversion on the non¬ 
linear function g. This is accomplished with any accuracy 
required in the analogue computer by making the non-linear 
function (j) have a very large (ideally infinite) transmission. This 
is equivalent to using a terminal error criterion L (e (AO) which 
imposes very large penalties for terminal errors and resembles 
the ‘penalty function’ approach suggested by Kelley 3 . The 
adjustment procedure is similar to that described for the problem 
of minimizing the terminal error. In fact, the computer has a 
built in flexibility which permits its use for either problem. The 
adjustment can be started with the assumption that the error 
cannot be reduced to zero. Set the transmission of cj) to a finite 
level and then adjust the parameters of the g unit in an attempt 
to satisfy the constraints. If this assumption is warranted, then 
at least one of the A* (or /<r ?: = A i //^“ 1 ) will be non-zero, since 
the optimum will result in at least one constraint being satisfied 
with the equals sign. However, if this assumption is not warranted, 
the implication is that all of the constraints are satisfied by the 
strict inequality and all the A ?: will be zero. In this case the 
transmission through the g unit will be infinite, the error will 
automatically be reduced to zero, and the effort F l will simul¬ 
taneously be minimized, since the computer will then be solving 
eqn (34). In practice, of course, the transmission of the g unit 
cannot be permitted to be infinite, but it can be made large 
enough to ensure that the terminal error is below any tolerable 
level. 

For the third problem, that of minimizing the number of 
steps to reduce the error to zero, use as the performance measure, 
F 0 , the number of steps that the error is non-zero. Since 
d F 0 /d e (n) = 0, n = v ,..., A, the optimum control law is also 
given by eqn (34); but here N is the smallest integer for which 
all the constraints are satisfied. It is noted that at the optimum N 
all the constraints could be satisfied by strict inequalities, 
although for any smaller N it is not possible to reduce the error 
to zero with any input which satisfies the constraints. 

The analogue computer could be used to minimize the 
number of steps by starting with a number of steps to go 
M > M min where M min is the smallest number of steps in which 
the error could be reduced to zero in the absence of any con¬ 
straints and is equal to the smallest integer p, for which the rank 
of [B AB is k. (It has been shown that \x < k, the 
order of the system unless the process is not controllable, in 
which case no such fx exists 8 .) At this setting of M it can be deter¬ 
mined whether all the constraints are satisfied with strict 
inequalities, in which case the transmission through g is infinite 
and the error is reduced to zero. If one of the constraints cannot 
be satisfied, M is increased by one and the process repeated until 
the value of M reaches results in the satisfaction of all the 
constraints. 

Recapitulation and Conclusions 

A straightforward calculus development has been used to 
obtain a system of algebraic equations which must be solved 


in order to determine optimum input sequence for a discrete¬ 
time process. If this process is to be controlled by the feedback 
method, wherein the present input is to be calculated m terms 
of the present error and available ‘effort’, then this computation 
must be performed in real time. A special-purpose computer 
has been displayed which it is feasible to use for implementing 
the required computation. The essential feature of this computer 
is the use of a feedback arrangement to solve a system of non¬ 
linear equations. Each component of the computer is ideally 
instantaneous, so that in principle the computation for a given 
setting of the parameter A* is instantaneous. In practice, one will 
have to wait until the transient, which will inevitably result owing 
to the presence of parasitic dynamic elements, decays sufficiently 
before its output can be observed. Moreover, the computer will 
require careful design to prevent the occurrence of unstable 
transients which could result when the transmission through 
the various units is high. 

Nevertheless, the analogue computer appears naturally 
suited to the solution of the problem, since there is no systematic 
method of obtaining an exact solution to eqn (27) or to eqn (34) 
on a digital computer. A digital computer could be used to obtain 
an iterative solution to eqn (27). A suitable iteration rule would be 


z (» + i) s=z (*) + ^{ z « + i4 /W ”v-i^[ i 4 JV v \Ae + g(z (t) ))~]} ^ 

where t indicates the iteration number. The function i jx must be 
selected to ensure convergence of z {i) , and, from a practica 
standpoint, that the convergence be rapid. The computation 
performed by such a computation technique turns out to be 
quite similar to the steepest descent techniques referred to 
earlier, and could probably not be performed in real time, unless 
the starting vector z ^ is close to the optimum. 

It is noted that the determination of the settings of the A ; do 
not change along an optimum trajectory, and in principle, could 
be left fixed after their initial determination. However, the use 
of feedback implies that errors will cause the process to deviate 
from the initially computed optimum trajectory, and the A* can 
accordingly be expected to change from step to step. But, it the 
deviation from optimum path is small, the adjustments require 
to obtain the correct setting of the A. ; can also be expected to be 
small, and whatever technique is used to obtain the mitia 
adjustments will work much more rapidly for subsequen ^ 
adjustments. As an approximation the A* could be fixed at then 
initial value and thereby greatly simplify the resulting computer. 
Such a simplification might be warranted in aerospace vehicles 
where the computer which makes the initial determination o 
the A, could be detached from the vehicle at launching. 
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DISCUSSION 

J. L. Shearer, Penn State University, University Park, Pa., U.S.A. B. Friedland, in reply 

Have you worked out a block diagram structure for the * unit ? As indicated in the text, the a unit is the non-linear device which 

transforms P u '{n)Z(n) into u(n) and is obtained by solving eqn. (16) 
for u(n). An example of this calculation is given following eqn. (30) 
and is accompanied by the graphical illustrations of Figure 3. 
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Summary 


Introduction 


This paper unifies and extends the ideas and techniques used by the 
authors in relation with the minimal time regulator problem. The 
system considered is a multiple input system described by a differential 
equation of the form x = F(x) + Buit) where u is either a PAM or 
PWM bounded control vector. This paper consists of five sections. In 
the first, the problem is carefully stated in a suitable general setting, and 
the concept of optimal strategy is defined; in the second, an optimal 
strategy for a multiple input PAM system with a completely control¬ 
lable linear plant is described fully, and a special case is discussed; in 
the third, the concept of equivalent optimal strategies is defined, and 
sufficient conditions for equivalence are indicated. In the fourth 
section equivalence is used to derive the optimal strategy of a PWM 
system whose linear plant has real and distinct eigenvalues, and finally 
in the last section, some extensions are mentioned. 


Sommaire 

Cette communication presente une synthese et unc extension des idees 
et des techniques preconisees par les auteurs pour la solution du pro- 
bleme du regulateur a temps minimal. Le systeme considere est un 
systeme a entrees multiples regi par une equation differentielle de la 
forme x = F(x) + Bu(t) ou u(t) est un vecteur de commande 
borne, a modulation soit par amplitude soit par largeur d’impulsions. 
La communication comprend cinq parties. Dans la premiere on 
delimite soigneuscment et judicieusement le probleme d’un point 
de vue general et on definit la notion de strategic optimale. Dans 
la deuxieme, on decrit en detail une strategie optimale pour un systeme 
a modulation par amplitude d’impulsions a entrees multiples- avec 
une installation lineaire complement reglable et on discute d’un 
cas special. Dans la troisieme, on definit la notion de strategies opti- 
males equivalentes et on enumere les conditions suffisantes d equi¬ 
valence. Dans la quatrieme partie, on utilise la notion d’equivalence 
pour etablir la strategie optimale d’un systeme a modulation par 
largeur d’impulsions dont Installation lineaire possede des valeurs 
propres reelles et distinctes. Enfin, dans la derniere partie, on indique 
quelques extensions. 


Zusammenfassung 

Der Aufsatz gibt eine Vereinheitlichung sowie Erweiterung der Ge- 
danken und Verfahren, die die Autoren im Zusammenhang mit der 
schnelligkeitsoptimalen Regelung verwendeten. Das betrachtete Sy¬ 
stem hat mehrere Eingange; es wird durch eine Differentialgleichung 
der Form x = Fix) + Bu(t) beschrieben, wobei der Vektor u die 
• pulsamplituden- oder pulsbreiten-modulierte Regelung mit Sattigung 
darstellt. Der Aufsatz besteht aus fiinf Abschnitten. Der erste Ab- 
schnitt enthalt das sorgfaltig dargestellte Problem in einer geeigneten 
allgemeinen Fassung und die Definition des Begriffes der optimalen 
Strategie. Der zweite Abschnitt beschreibt eingehend die optimale 
Strategie fur ein pulsamplituden-moduliertes System mit mehreren 
Eingangen bei vollig regelbarer linearer Regelstrecke; ein Sonderfall 
wird betrachtet. Im dritten Abschnitt wird der Begtiff der aquivalenten 
optimalen Strategie festgelegt und ein Hinweis auf hinreichende Be- 
dingungen fur die Aquivalenz gegeben. Der Begnff der Aquivalenz 
wird im vierten Abschnitt dazu benutzt, die optimale Strategie fur ein 
System mit Pulsbreiten-Modulation abzuleiten, dessen lmeare Regel¬ 
strecke reelle und verschiedene Eigenwerte hat. SchlieBlich sind im 
letzten Abschnitt einige Erweiterungsmoglichkeiten erwahnt. 


One of the fundamental problems of engineering is that of im¬ 
proving the performance of a given system. In the early days of 
automatic control, this consisted almost exclusively of compen¬ 
sation techniques. Under the impact of decision theory and 
Wiener’s 1 and Kolmogorov's 2 work on the minimization of 
mean squared error, strong interest developed in the mathe¬ 
matical formulation of optimization problems. Later McDon¬ 
ald 3 emphasized the importance of the constraints on the con¬ 
trols, formulated the minimum time problem, and introduced 
the idea of switching curve. It is now customary to state an 
optimum control problem in the following way. Given a system 
described by a set of differential or difference equations, a re¬ 
stricted set of (admissible) controls, and the requirement that 
the system be transferred from its initial state to a set in its state 
space, find an optimal strategy such that it specifies an admis¬ 
sible control fulfilling the requirement and minimizing a cost. 

In the last ten years a large number of papers appeared in the 
field of optimal control. They have culminated in a number of 
important results such as Bellman’s principle of optimality , 
Pontryagin’s maximum principle 5 , the bang-bang principle 
emphasized by La Salle 6 - 8 , the concept of controllability 9 and 
canonical structure 10 . At the same time other investigators 
sought to implement the ideas of optimal control by constructing 
optimal switching surfaces 11 ’ 12 , digitizing the maximum prin¬ 
ciple 13 , devising various iterative techniques 14 , 15 , and using the 
calculus of variations 8 >, 16 > 17 . 

In parallel with the work on continuous systems, rapid de¬ 
velopments occurred in discrete systems. Essentially there are 
two approaches to such problems. In the first Krasovski! 18 ’ 
and Tsypkin 20 showed how the problem could be handled either 
by a reduction to the Krein L-problem or else by the maximiza¬ 
tion of a functional by the calculus of variations. The second 
approach is analogous to techniques used on mazes and discrete 
state machines 21 and to dynamic programming. This method 
consists of partitioning the state space of the system into sets 
Rn, (IV = 1,2,...), from which a transition to the desired state 
is possible in N steps and no fewer. The minimal step paths, 
hence the optimal controls associated with these, connecting the 
initial state with the desired state can then be found either by 
inspection or by making use of the algorithms used in the con¬ 
struction of the sets Rn • Consequently, these can be used to 

define the optimal strategy. . . 

This method was used by Kalman 22 to consider a minimal 
time problem of saturating PAM discrete regulator systems. 
Later Nelson 23 considered a similar problem for the PWM case. 
The authors of this paper considered the same problem as 
Kalman, introduced the concept of critical surface, expressed the 
optimal strategy in terms of ‘distance’ measurements to this 
critical surface, and showed the relationship between the discrete 
case and the continuous case 24 . These early results were gener¬ 
alized and extended to more complicated situations and 
also applied to PWM systems 28 . An important new idea was the 
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introduction of an equivalence relation between systems 24 . This 
equivalence can be used to construct an optimal strategy for one 
system from the known one of another. This technique has been 
successfully applied to obtain the optimum control for PWM 
systems 24 , and for systems with non-linear plants 31 - 32 . 

This paper considers from a unified point of view and extends 
recent results pertaining to the minimal time regulator problem. 
The paper consists of five sections. In the first section the prob¬ 
lem is carefully stated in a suitable general setting, and the con¬ 
cept of optimal strategy is defined. An optimal strategy for a 
multiple input PAM system with a completely controllable 
linear plant is described fully in the second section, and a special 
case is discussed. In the third, the concept of equivalent optimal 
strategies is defined, and sufficient conditions for equivalence 
are indicated. In the fourth section, equivalence is used to derive 
the optimal strategy of a PWM system whose linear plant has 
real and distinct eigenvalues, and finally in the last section, 
some extensions are mentioned. 


Statement of the Problem 

All the systems considered in this paper have a block dia¬ 
gram representation shown on Figure 1 . In each case the system 
consists of a plant, a modulator, and a computer. 


symbol u will be used for either modulator output. The modula¬ 
tion laws for the two modulators are given as follows: 

Let in both cases C[=±l, 0<o4<l for all i and k 

PAM : ut(t) =CXM for (k~Y)T<t<kT (3) 

PWM: uY(t)=fc M * OT ( k ~ 1 ) T - t<( - k ~ 1 + a k) T 

(0 for (/c—1 + 0 £‘) T<t<IcT ^ 

where in both cases, k = 1,2, 3,...; i = 1, 2,..., m. M is the 
maximum amplitude of the modulator pulse output, T is the 
sampling period. Since, during any sampling period, the modu¬ 
lator output is characterized by m numbers, let the vector 
Pk — col (£&&&, £|^f> • • be introduced to define the 

modulator output during the interval (k - 1) T < t < kT. 
Throughout the paper, for all f s and k% Cl = ± 1 and 0 < oc l 
< 1. Any control defined by such a vector and by either modu¬ 
lator law (3) or (4) is called admissible. Finally, let u [0i nt] be a 
vector-valued function equal to the control u (t) for 0 < t < NT 
and equal to 0 anywhere else. It is important to note that there 
\n e_ f°“ one correspondence between M[ 0} W!T] and the sequence 
\Pm~i- The set of all finite sequences jwhere the com¬ 
ponents of each /?# satisfy the constraints specified above is 
called the sequence space S. 



Figure 1. Block diagram of a representative system 


The plant is either a linear or a non-linear time invariant 
dynamical system 9 - 10 with m input channels, which is completely 
controllable * with respect to 0 by the modulator output and is 
described by a differential equation whose solution is unique 
with respect to initial conditions and inputs. In general, the 
differential equation has the form 


x = F(x,u ) (j 

* == colfxj, x 2 , ..., x n ) is the state vector, u(t) — c< 

lAiU), u,,(t) . u m (t)] is the m-channel modulator output an 

t is a n X 1 vector valued function, Lipshitzian with respect to 
and such that F (0,0) = 0 3 ». In the particular case when tb 
plant is a linear dynamical system, the differential equatio 
takes on the form: 

x—Bx+Eu(t) (2 

whereB isanx« constant matrix and £ is a constant n x t 

comnw matr f ™*. columns The condition c 

omplete controllability imposes additional restrictions on th 

p ant which must be worked out for each particular case. 

t he modulator may be either a pulse amplitude (PAM), o 
a pulse width (PWM) modulator. When it is necessary to' dis 

bv^andTPW^ 6 m0dulators ’ a PAM is designate! 

by u and a PWM output is designated by u w ; otherwise th< 

oriJnlfWT ' SSa ! d 1° be com P letel y controllable with respect to the 

stefnthisstatemT ^ the f exists an admissible control whicl 
steers tms state to the ongm m finite time. 


The computer generates the p k ’s at t = (/c— 1)T, k = 1, 
2,..., on the basis of x k - lt the observed value of the state at 
that instant, in such a manner as to transfer the system from 
x k-i to the desired state 0 in the smallest number of sampling 
periods possible. Let this number be N— k + 1. It may be as¬ 
sumed for the purpose of the following that the computer gener¬ 
ates the whole vector sequence at t = (k- l)/ feeds 

the vector/?,, to the modulator and stores the rest of the sequence. 
At t = kT, the computer feeds ji k+1 to the modulator, etc. In 
practice, the computer needs only to compute the first element 
of the sequence at each sampling period. 

For the systems under consideration, an optimal strategy is 
a function, /, whose domain is the state space E n and whose 
range is the sequence space 5, with the following two prop¬ 
erties First, for any x in £»,/(*) is a finite sequence, say of 
length N, such that the modulator output defined by the se¬ 
quence/^) transfers the state x to the zero state, 0, in N sam¬ 
pling periods; and second, there is no sequence of length 
smaller than N which defines a modulator output capable of 
transferring x to 0 in less than N sampling periods. 

The minimal time regulator problem can be stated as fol¬ 
lows. Given a completely controllable system such as the one 
shown on Figure 1, together with its plant dynamics and its 
modulator law, find an optimal strategy for that system 

For any integer N, H N is the set of all initial states that can 
be transferred to the origin by an admissible control in N sam- 

pling periods and no fewer. Let UR,„ i.e., it is the set of 

all states that can be transferred to the origin by an admissible 
control in no more than N sampling periods. The sets can 
be used to characterize an optimal strategy. A mapping/ of the 
state space E n into the sequence space S' is an optimal strategy 
if and only if, for all N, and, for all x e R N , the first element of 
tbe A S t, quenc ® sa y defines a modulator output, «m T] 
u Cri)j which takes the state x at time 0 to a state y at time T 
such that y e u^t] = H(ft) is said to be optimal. 
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An Optimal Strategy for a PAM System with Linear Plant 

Suppose the plant is a linear, time invariant, dynamical 
system and suppose its state equation is given by eqn (2). To 
avoid trivialities, the rank of E is assumed to be m. Let the control 
«°(r) be of the form (3). In view of the form of the modulation 
law, the system in terms of its state transition equation is now 
described. Let x k be the state at time kT, then for k = 0,1, 2,... 

m 

X k +i —Ax k -h Yj (5) 

*t 

where A = exp BT, and dj (t) — Me B(f s) ejds 

Jo' 

0 = 1,2,...,*!). 

Iterating (5) N times, one obtains 

N m 

x^A N x o+X Y tXA N - k dj(T) (6) 

k=l j =1 

If one defines 

r k ( t )=~A~ k dj(t) (j = l,2,...,m;/c=l,2,...) (7) 

then, if xjv = 0, one obtains* 

N m 

X 0 = z z tidri (8) 

k=l j- 1 

An optimal strategy for such a PAM system is now de¬ 
scribed. First, controllability is considered 26 . 

Theorem i—The system (2) with modulation law (3) is com¬ 
pletely controllable by admissible controls if and only if the set 
of mn vectors S mn = {r] a 1 < k < n, 1 < j < m ) s P ans 
state space E n and the eigenvalues of A lie in the closed unit 
circle |/i[ < 1. 

If one eigenvalue of A, say A l9 lies outside the unit circle, then 
a state x is completely controllable by admissible controls pro¬ 
vided the projection of x on the invariant subspace associated 
with A-jl lies within an open set which is determined by A and E. 

The optimal strategy can be described in terms of m hyper¬ 
surfaces ^‘called the critical hypersurfaces 26 ’ 27 . With the yth 
component of the control vector, (1 < j ^ IS associated a 
critical hypersurface The critical hyper surfaces satisfy the 
following intersection property. For j = 1, 2,.. m 9 any straight 
line parallel to r{ intersects at one and only one point. There¬ 
fore given any state x, the m numbers 2^(x) defined by the 
relation x = 1/(X) r{ 4- y 3 where y 3 e are uniquely defined. 
Geometrically X 5 (x) measures the ‘distance’ between * and % 
provided ‘distance’ is measured in the direction of v[ and the 
Euclidian length ||r||| is taken as the unit of lengthy If sat y is 
defined as y/\y \ whenever the real number y is outside [—1,1] 
and sat y=y is put for all other y, then the vector <j>(x) = 
col [sat Aj(x),..sat A w (x)] defines an admissible control, for 
that x. Let D be the nxm matrix whose columns are the vectors 
d 3 {T\j = 1, 2, ..., m. It can be shown 27 that if x 0 e Rn at t = 0, 

* Throughout the Section 2 of the paper, we shall always write 
4 for r^T). 


then Ax 0 + D cj>(x)eR N - x at t =T. In other words, the control 
defined over [0,71 by the vector $(x), i.e. u [(j>(x)l is optimal. 
Since this holds for all n and all x 0 in Rn, the function <j> can be 
used to define an optimal sequence for x 0 , namely {(j)(x 0 ), 
4>(*n- i)} where = Ax k + D 4>(x0. In other 
words, the sequence {4>(x 0 ), 0(xi)> ..., <j>(x N - 1 )} is the value 
taken at x 0 , namely /(x 0 ), by an optimal strategy/. Going back 
to the block diagram of Figure 7, it is seen that the function <p 
specifies an optimal programme for the computer. The process 
of evaluating (j) (x) by ‘distance’ measurements at the beginning 
of each sampling period is illustrated by Figure 2. It shows the 
state space of a second order system with two inputs; the 
critical surfaces tf 1 and ^ 2 are shown and the determination of 
(j>(x) is illustrated. 

Consider the particular case where the plant has a single 
input u 1 . For each A, R# is a convex polyhedron in E n . For 
each N = 1, 2,..., R& can be obtained from R^- x as follows. 
Translate each vertex of R ^~i by ± rj, the convex hull of this 
new set of points is R^ In other words, some faces of R N can be 
obtained from those of R^-i by translating them in the direction 
fir 1 where the sign of & is selected in such a way that the 
translation occurs in an outward direction with respect to R^-i- 
If, in addition, the single input PAM system is such that A 
[in (5)] has real distinct eigenvalues, then the hypersurface % 
has special properties that lead to a simple analogue computer 
set-up to evaluate <£(x) 25 . This has been tested experimentally 11 
and it was found that the system was not particularly sensitive 
to changes in the parameters. 

Going back to the m input plant, in order to see what 
happens to the optimal strategy as the sampling period T-> 0, a 
description is now given of how R^ is obtained from R N -For 
notational convenience, let R^ = Rn,o an(i r n = R N<m- Con ~ 
struct iteratively the sequence of sets j~v Since R~h ~ 0 is a 

polyhedron in E n , it will be seen that each one of the sets of the 
sequence is a polyhedron. is obtained from Rfr,, as fol¬ 

lows. Translate each vertex of Rnj-i by ± r v> tbe convex hull 
of this new set of points is the polyhedron R^j. Performing this 
operation m times, R^ m — R$ is obtained from Rn,o ~~ r n~: i* 
Now, let x e R N , v but x $ R N , y -i- The scalar p y N is uniquely defined 



Figure 2. This figure shows the two dimensional state space of a two 
input system. Note the sets R[, 7&... and the critical curves W and V . 
X 1 (x) andX 2 (x) are the ‘ distances ’ of the state x 
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by the following condition*. /?$ is the smallest in absolute value 
of v such that x = vr} + y where y e b R^. By the preceding 
construction, \@h\ < 1 . Proceeding in this manner until one 
reaches R' 0 = [0] , one gets 

N — 1 m y 

*=E E PM+ 1 flW (9) 

k=l ./=1 j=l 

where ftl’s are uniquely determined and belong to [- 1 , 1 ]. This 
unique representation of x is called the canonical representation 
of x. It is of interest to note that (p (x), defined earlier in this 
paper has f>\, ..., as components. An important property 
of that representation is that, for each j, the sequence 
has, at most, n element that are in (— 1 , 1 ), and all others are 
either 1 or — 1 . Since a PAM system is being considered it 
means that differs from +1 or —1 at most over a set 

of measure nT. Therefore as T 0, will be bang-bang. 

In other words, almost everywhere in the state space (p (x) has 
in the limit, components equal to 1 in absolute value. 

Equivalent Optimal Strategies 

Let us now go back to some general considerations and ex¬ 
amine the concept of an optimal strategy. Let / be an optimal 
strategy for the system under consideration. For each x, the 
sequence / (x) is a sequence of vectors {p k } k==1 (where N depends 
on x) such that if, at (k - 1) T, (Ic = 1, 2,..., N\ the input to the 
plant is the modulator output defined by j$ ]c then x (NT) = 0 
and no admissible control exists which transfers x 0 to 0 in less 
than N sampling periods. 

In view of the uniqueness theorem of the theory of differen¬ 
tial equations and of the assumptions made on F (x, u ) in (1), 
Xi ^ x 2 implies that f(xf) #/(x 2 ). Therefore / is a one-to-one 
mapping of the state space E n onto f(E n ). Therefore the inverse 
map,/" 1 , exists :/” 1 maps/ (E n ) onto the state space E n . This 
mapping is one-to-one since the system is completely con¬ 
trollable. Note that / (E n ) is the set of all optimal sequences 
associated with the optimal strategy/. 

To illustrate these ideas, consider the PAM system and the 
state x 0 given by eqn ( 8 ). The mN numbers V k define a sequence 
sjv of N elements each one of which is an /?z-vector. Thus the 
right-hand side of ( 8 ) associates with each such sequence sn a 
state x. In other words, by a straightforward analysis of the 
system, a function g is obtained, mapping the sequence space S 
onto the state space E n , 

N m 

<K%)= I I 044 

fc = 1 j ~ 1 

Introducing a definition, {p k } k==1 is said to be a minimal 
sequence if this sequence transfers some state to 0 , and if no 
other sequence can do it in less than N sampling periods. 

The function g can be used to construct an optimal strategy 
as follows. First the domain of g must be restricted to the set of 
all minimal sequences. Then, from the properties of F in ( 1 ), the 
inverse images of all x £ E n are disjoint sets, so that if, from each 
inverse image one and only one minimal sequence is picked 
arbitrarily, the resulting restriction of g , say the function g, is 
one-to-one and finally, g~ x is an optimal strategy. 

* fiji is defined as the/th component of the P k defined earlier in this 
paper, i.e., 0 * = 


Let x 0 be any initial state, and let/(x 0 ) = {P k } k==1 be the 
sequence assigned to x 0 by the optimal strategy. Let u (f(x 0 )) be 
the control defined by the sequence /(x 0 ). If the system is now 
steered by the control w [/(x 0 )], it is clear that the sequences 
{/W*- 2 > {Pk}k- 3 » • • *5 {Pn} 9 will be minimal for the states x ls x 2 , 

x x - l9 and that xn = 0. Since a given state may have more 
than one minimal sequence, it is convenient to assume that the 
optimal strategy assigns precisely these sequences to these states 
and that this is true for all x 0 6 E n . An optimal-strategy which 
satisfies this condition is said to be consistent. 

Definition —Consider two systems A and B, with state 
spaces Ea and Eb , with state vectors x and y, with consistent 
optimal strategies f A and f B and inverse functions /f 1 and/ 5 ” 1 . 
The systems A and B are said to be optimal-strategy-equivalent 
iff a (E a ) =f B (E b ). 

Discussion —Consider any x 0 e E A , since f A (E A ) = f B (E B ), 
there exists a y 0 eE B such that /5 (y k ) = f A (x k ), k = 0,1, N, 
where N is the length off A (x 0 ). Thus, if the transients under 
the optimal controls u [ f A (x 0 )] and u [f B (x 0 )] are observed in 
the sequence space, systems A and B are indistinguishable. 

Now apply this concept of equivalence to derive an optimal 
strategy for a system B from the known optimal strategy of a 
system A. Consider a system A with an optimal strategy f A and 
its inverse/T 1 . Let B be any other system, in the class considered, 
for which it is necessary to construct an optimal strategy, and 
let g B be the function mapping the sequence space S on to E B , 
such that if sn transfers the state y to 0 in N sampling periods, 
then g B ($n) ~ y , (see Figure 3 ). 


Ea S Eb 



Figure 3. Illustration of the mappings necessary for the definition of 
equivalence. The set S is the set of all optimal sequences under the 
optimal strategy f A 

Clearly, the state space E A can be mapped into the state 
space Eb by means of the function g B f A . Now let the state of A 
be x and the state of B be y. If 

^5/4 (^4) = ( 10 ) 

^n/4(^i)^^n/4fe) for all x 1? x 2 such that x 1 ^x 2 (11) 

then the function g B f A is a one-to-one mapping of E A on to E B . 
Therefore its inverse exists and one can construct a function f B 
on the state space of B, such that for all xeE A : 

fB(y) = f A (x) where y = g B f A (x) ( 12 ) 

Since f A is optimal, its inverse f~2 exists and, from (12 ),fff B is 
the inverse of g B f A and it maps E B on to E A . 

Theorem 2—Let be the set R N for system A and let R§ 

be the set Rjy for system B, where Rjy was defined earlier in the 
paper. If for all N = 1, 2,..., g B f A (R$) = r§ 9 then the func- 
tion f B defined by (12) is an optimal strategy and system A and B 
are equivalent under the optimal strategies f A and/ 5 . 
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The usefulness of this theorem lies in the fact that the 
function 0b is usually easy to determine, hence once the assump¬ 
tions of the theorem are satisfied it is easy to derive an optimal 
strategy for a system from the known optimal strategy of 
another system. 

An Optimal Strategy for a PWM System with a Linear Plant 

In this section single input PWM systems arc considered, 
for which (2) reduces to the form 

x = Ax + e 1 Ui(t) (13) 

A is a n x n matrix with non-positive real distinct eigenvalues. In 
this case, sn = {££<4}f= i is a sequence of scalars and the 

function g 1 ,(SN) = y j Clrl(oclT). It is extremely difficult to 

7c-1 

construct independently an optimal strategy for this system and 
hence, the optimal strategy f w is found from the optimal strategy 
f a for the system, 

x = Ax J re 1 u a 1 (t) (14) 

where A is the same as in (5). 

Consider the optimal strategy j a . In the process of con¬ 
structing the sets iV = 1, 2,..., for the PAM systems, it was 
found that minimal sequences are constructed for all the points 
in the state space in a unique manner. To clarify this statement, 
consider any point x 0 in Rfr, N = 1,2,.... Let the vectors 
f i oc i vl (T\ Ck = ± 1, 0 < <4 < 1, be added to the boundary 
of R '$in a scanning motion until it intercepts the given 
point x 0 . Now let the vector Cv-i i r N-i (T), £a : -t = ± 

0 < < i be added to the boundary R' S - 2 in a scanning 

motion until it intercepts the point of contact of the vectoi 
with the boundary of Ki- 19 and so forth, using 
vectors £\(x\r\(T), k = N— 2, N— 3,..., 1, Cl = ± L 0<oc k < 1, 
which are added to the boundaries of the sets R' k ~i- This process 
generates a sequence of length N for the representation ot the 
state jCq, and, since the state Xq is by hypothesis in R jw this 
sequence is a minimal one and thus the procedure defines an 
optimal strategy. 

In practice it is necessary to compute only the first membei 
of the sequence f a (x 0 ) for any given state x 0 in ordei to ensure 
minimal time control. This is much easier to compute than the 
whole sequence f a (x 0 ) and may be done by addding the vector 
- Ci A r i CO, Ci = ± 1 to the point x 0 , where A > 0 is a scalar, 
and by adjusting A until the vector [x 0 — Ci A COl touches 
the J. If the resulting > 1, oc\ c\ = Ci, and if the resulting 
A < 1, then Cl c\ = Cl A. Thus, oc\ is a function of the distance 
of the point from the c €\. It is clear that the remainder of the 
sequence for any given point could be computed by iteration. 

Now some useful properties of the vectors r\ (t) are described. 
For the system (13) it can be shown that 

r\ (/) = r{[t + (k -1) T] - r\ [(Ac- D T] (15) 

Theorem 3 —Let c„ be a chord on r{ 01 0 < t < oo, such 
that c iS = r{ Of) ~ r\ Oil U > U- Consider any set {c hJ JUi 
of n such chords where t tl < t si < t i2 < t j2 ,< ... < On- the 
set i forms a basis for the space E w . 

Introducing a mapping of the sets R% into E tv , let R$ be the 
set Rn in E a and let R§ be the set Rn in E w . Let the parametrized 
arc traced out by the vector ft r\ (4 T) for 0 < «* < 1 be 
denoted by ft 4, k = 1, 2, ...,ft = ± 1. Consider some 


sequence sn e S, then, to map this sequence into E a , the sum 

f ]Clclrl(D is formed, i.e., geometrically, the parametrized 

straight line segments rj c (T), k = 1, 2,.... N, are put together 
in the manner prescribed by the sum. In order to map this same 

A 7 

sequence sn into E w , the sum r[. (oc\ T) is formed, 01 , again 

speaking geometrically, the parametrized arcs C\L)n k = 1,2,. .. 
are put together in the manner prescribed by the sum. Thus, in 
order to map the sets R% into E w by means of the function 
g w f a , the arcs L k are put together in E w in the same canonical 
representation as the vectors (T) were put together in E a to 
form the sets R$. It is clear that there is no ambiguity as to 
direction, since the vectors r\ (T) form chords on the arcs L k . 
Consider what happens when the arc Ljs? t N — 1, 2, ..., is 

N -1 

added to the map of one face of the convex polyhedron U Rf. 

Y-l k—0 

The face of U R a k in E a is at most an (n-l)-dimensional poly¬ 
hedron and its map in E w is formed by a combination of at most 
(n-1) of the arcs C\L\, ft = ± b k = 1,2,..., N- 1. Hence, 
when the arc Cn^Cn = ± 1 is added to any point of the map 
of the face of R^ lt it follows from Theorem 3 that it does not 
intersect this map at any other point. It also follows from 
Theorem 3 that if this addition of the arc Cn 4v is done in a 
continuous scanning motion, a hypervolume is traced out, each 
point of which is intercepted only once by the arc Cb Av- Si nc ® 
the vertices of the map of R§ are the same as the vertices of 4v, 
if added outwards to one face of the map of Rb-i, the arc L N 
will not intersect any other face of the map of R §- x . Further¬ 
more, it is clear that if the mapping process is continued, the 
maps of the sets Rb will fill the space E w . Hence it follows that 
g w f a (E a ) = E w and that given any x lt x 2 in E a , g w f a (x x ) # 
g w fa (x a ) if x t # x 2 . It is therefore possible to define a function 
fw on E w such that 

fwidwfa ( X o)) = fa ( X o)> X 0 6 

This is the function )'b defined in eqn (12) for the general case. 
Now, it will be observed that as the arcs ft- 4- 4 = i 1 are 
put together to form the maps of the sets R%, they produce 
hypervolumes which contain all points expressible in the form 

2 ft 4 (* 1 T), Cl = ± b 0 < «| < 1. It follows that the maps 

of 1 the sets R'f are the sets Rf and hence the sets R y are 
invariant under the map g w f a . It may therefore be concluded 
that the function f w defined in (16) is indeed an optimal strategy 
for the PWM system. 

Consider now the distance function required for the com¬ 
putation of ft 4 for the PWM system. The map of E a on to E w 
by means of the function f w f a is clearly, one-to-one and ^con¬ 
tinuous. Hence, the map of the critical hypersurface (©,,) from 
E a into E w results in a critical hypersurface (Aft in E w with the 
same properties with respect to f w as has with respect to/ a . 
Furthermore, it is clear that all the states x of E w , for which 

f i jV = 1,2, ..., is such that 0 < oc\ < 1, 

Jw W “ t^7c iY ’ ’ . r r „1 r^.1 r r\ 

are assigned by f w a representation of the form x - C a *T 1) 

+ y l9 where y x is some vector such that y x e and Ci = ± L 

0 < oc\ < 1 . 

It therefore follows that in order to compute the first term 
ft tx\ of the sequence /„ (x 0 ) for the state x 0 , one forms the sum 
[x Q - 0 < t < oo, and one finds the value of t for which 
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tfo ~ r i (/)] e Then, if / > 1, Cl ocj = fj, and if / < ], 
4 'j — Cl t. Again, it is clear that the remainder of the minimal 

sequence for the point x 0 could be computed by an iterative 
technique. 

A Few Extensions 

Consider the problem of transferring in minimum time an 
initial state x 0 to a fixed target state x t by an admissible control. 
The set of all admissible controls which transfers x 0 to x ti say 
A(^o : ^f)j may be empty. The problem of existence of such 
admissible controls is the analogue of the controllability 
problem in the first section of this paper. If the target state x t 
belongs to the set Q, where Q is the set of all states reachable 
from the origin by admissible controls of a completely con¬ 
trollable system, then A ( x 0 ; x t ) is not empty for any x 0 in the 
state space. 

Consider a system described by eqn (5). For any x t , for which 
A (x 0 ; x t ) is not empty, the problem reduces to the minimal time 
regulator problem. The procedure is as follows. First, determine 
the minimum N, say M, such that (x 0 - A~ M+1 x t ) $ R' M - X but 
(x 0 — A M x t ) e R'm ; secondly, consider the state x 0 — A~ M x t 
as the initial state of the system for the minimal time regulator 
problem, thirdly, the optimal strategy for transferring x 0 to x^ 
in M sampling periods is obtained iteratively as follows. The 
optimal control for the first sampling period is the optimal 
control for transferring the state x y - A~ M x, to 0. namely 
^(x 0 ~^-^x^). 

Note that the resulting sequence is not the optimal sequence 
associated with transferring x 0 — A~ M x t to 0. 

Earlier in this paper an optimal strategy for a PWM system 
was derived from that of a PAM system, each system having 
identical linear plants. However, for two systems to be equivalent, 
the plant of the first need not be identical with the plant of the 
other system. In fact 31 , a second-order PWM system with a 
linear plant, say, 


*1 


0 l 




'o' 
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4- u 1 (t) 


x 2 


0 —a 
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is optimal strategy equivalent to the second order PWM system 
with a non-linear plant 
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provided g(x 2 ) is odd and has a continuous derivative g'(xo)> 0 
for all x 2 . 
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DISCUSSION 


Ya. Z. Tsypkin, Institute of Automatics and Telemechanics , Moscow 

I — 15, Kalantskovskaja 15a, U.S.S.R. 

I should like to ask the following questions: 

(1) What is the nature of PAM and PWM optimal processes? 

(2) Did you compare your proposed optimal systems with similar 
systems using linear control laws? 

(3) What is the nature of the processes in such an optimal system 
when it is not excited by the presence of an initial condition, but by 
means of an external excitation such as, for example, a monotonic 
input ? 

C. A. Desoer and E. Polak, in reply 

We thank Professor Tsypkin for his interest and his stimulating 
questions, which we answer as follows: 

First, the control signal generated by our proposed optimal strategy 
has the following properties: (1) not more than n of the oc/c’s are less 
than 1, and all other ot*’s are equal to 1; (2) for real eigenvalues, the 
sequence has, at most, n — 1 sign reversals; (3) for complex eigen¬ 
values, given any number, say M, initial states can be found such that 
the associated control signal will have more than M sign reversals. 

Second, we have not obtained analytical results which allow us to 
compare conveniently our optimal systems with similar systems 
governed by linear control laws. However, our algorithm to construct 
the sets Rn can be used to compile a control table for an arbitrary 
bounded portion of the state space. This table may then be used to 
obtain an average cost for the system. This figure of merit may then 
be used for comparison with other control laws. 

Third, a second-order PAM system has been simulated on an 
analogue computer. Its behaviour with respect to standard inputs, 
noise inputs, and parameter perturbations were found to be satis¬ 
factory from an engineering point of view. Our experimental investi¬ 
gations are at present being expanded to include PWM systems and 
systems of higher order. We hope to be in a position to report on our 
experimental results in the near future. 


R. G. Mariouw, Ministry of Defence, Juliana van Stolberglaan 381. 
Den Haag, Nederlands 

In their extremely interesting paper on minimum time discrete regula¬ 
tors, Professor Desoer and his co-workers consider the optimization 
of a system described by the equation 

x = Bx + Eu(t) 

including a modulator of either the P.A.M. or P.W.M. type. 

However, an important practical case obtained, for instance, in a 
controlled rectifier modulator, is that of pulse phase / amplitude 
modulation (PPAM) in which there is a time delay of the form 

H{t-(K\u\ + a) T } 

associated with the control vector u ( t ), whilst furthermore there is a 
constraint on the maximum modulus of U ( t ). In my opinion, the 
control matrix E will then be no longer constant. 

Can the authors please comment on this question and possibly 
show the way in which an answer might be obtained ? 


C. A. Desoer and E. Polak, in reply 

We thank Mr. Mariouw for his discussion and for bringing to our 
attention this practically motivated modulation scheme. Unfortuna¬ 
tely, his description does not define precisely the modulation scheme 
involved. However, since controlled rectifiers are mentioned, the 
modulation scheme might be such that the pulse width is proportional 
to the input and the leading edge is delayed proportionately to the 
input. In our analysis of PWM systems, the leading edge was fixed 
and the lagging edge was delayed proportionately to the input. 
Because of the similarity of the resulting non-linear difference equa¬ 
tions, it seems, therefore, that some of the techniques of our paper 
should apply to this problem. 


On the Roots of a Real Polynomial Inside the Unit Circle and a 
Stability Criterion for Linear Discrete Systems* 

E. I. JURY 


Summary 

Necessary and sufficient algebraic conditions for the roots of a real 
polynomial to lie inside the unit circle are given in table form. In this 
form, the constraints are obtained only by evaluation of second-order 
determinants. The connection and identity existing between the sta¬ 
bility criterion established in this paper and that of a previously ob¬ 
tained criterion 1 are discussed. The usefulness of the table may be found 
in those cases where the coefficients of the real polynomial are given 
in numbers. It is similar to Routh’s table obtained for the continuous 
case. 

Conditions on the numbers of the roots inside, outside, or on the 
unit circle are also discussed, within the cases when the determinants 
are zero or non-zero. Also, necessary and sufficient conditions are 
formulated for all the roots to be inside a circle of radius a less than 
unity, and also the conditions when the roots are to lie between plus 
and minus unity in the z plane. 

Various examples of discrete systems are presented which illus¬ 
trate the applications of the new stability criterion as well as the other 
conditions formulated in this paper. 

In concluding the paper, various analytical stability criteria applied 
to linear discrete systems are enumerated and compared, with empha¬ 
sis on the advantageous applications of each. 

Sommaire 

Les conditions algebriques necessaires et suffisantes pour que les 
racines d’un polynome reel se trouvent a Finterieur du cercle unitaire, 
sont donnees sous la forme d’un tableau. Sous cette forme les limita¬ 
tions sont obtenues par revaluation d’un determinant du 2e ordre 
seulement. La relation existant entre le critere de stability decrit dans 
ce rapport et un critere deja connu, est examinee. Ce tableau est utile 
dans les cas oil les coefficients du polynome a analyser sont connus 
sous forme numerique. II rejoint le critere de stability de Routh dans 
le cas continu. 

Le nombre des racines situees a Finterieur, a l’exterieur et sur le 
cercle unitaire, est discute et cela aussi bien dans les cas oil les 
determinants sont nuls, que lorsqu’ils sont differents de zero. En outre, 
les conditions necessaires et suffisantes sont formulees pour que 
toutes les racines soient a Finterieur d’un cercle de rayon a inferieur 
a Funite; il est egalement indique quelles sont les conditions pour 
que les racines soient comprises entre plus ou moins un dans le plan z. 

Differents exemples de systemes discrets sont indiques pour illustrer 
Fapplication de ce nouveau critere de stability et les autres conditions 
decrites dans ce rapport. 

En conclusion, differents criteres de stability analytiques applicables 
a des systemes lineaires discrets sont enumeres et compares en mettant 
en evidence les avantages et les inconvenients de Fapplication de 
chacun d’eux. 

Zusammenfassung 

Die notwendigen und hinreichenden algebraischen Bedingungen dafur, 
dafi die Wurzeln eines reellen Polynoms innerhalb des Einheitskreises 
liegen, werden in Tabellenform angegeben. Zur Ermittlung der Stabi- 
litatsbedingungen sind dabei nur Determinanten zweiter Ordnung zu 
Ibsen. Die Zusammenhange und die Entsprechung zwischen dem hier 
aufgestellten Stabilitatskriterium und einem fruher erhaltenen Kri- 
terium 1 werden dargestellt. Diese Tabelle mag in solchen Fallen niitz- 


lich sein, in denen die Koeffizienten des reellen Polynoms zahlenmaBig 
vorliegen. Sie ahnelt der Routhschen Tabelle fur den kontinuierlich 
arbeitenden Fall. 

Bedingungen fur die Anzahl der Wurzeln innerhalb, auBerhalb 
und auf dem Einheitskreis werden fiir die Falle, daB die Determinanten 
Null oder von Null verschieden sind, besprochen. Es werden auch die 
notwendigen und hinreichenden Bedingungen dafur formuliert, daB 
sich alle Wurzeln innerhalb eines Kreises mit dem Radius a < 1 be- 
finden und dafur, daB die Wurzeln in der z-Ebene zwischen +1 und 
— 1 liegen. 

An Hand von verschiedenen Beispielen diskreter Systeme werden 
die Anwendung dieses neuen Stabilitatskriteriums wie auch der ande- 
ren hier formulierten Bedingungen gezeigt. 

AbschlieBend werden verschiedene analytische Stabilitatskriterien 
fur lineare diskrete Systeme aufgezahlt und mit besonderem Hinweis 
auf die jeweils gunstigsten Anwendungsfalle miteinander verglichen. 


Introduction 

A useful criterion to obtain the number of roots of a real 
polynomial in the unit circle has been long recognized and 
desired. The early work of Schur-Cohn 2 , Fujiwara 3 , Marden 4 
and others is an indication of the importance of such a criterion. 
Since heretofore no simple criteria had been obtained, many 
authors in the past have used transformation to a different plane, 
in which several criteria were either available or readily obtained. 
Among the general transformations, the bi-linear and the unit 
shifting are sometimes used. The advantage of the bi-linear 5 
transformation, which maps the inside of the unit circle into 
the left half of the W plane, is the ready use of the Iiurwitz, 
Lienard-Chipart, or Routh tables. However, this bi-linear 
transformation involves complicated algebraic manipulations 
which increase in complexity as the order of the real polynomial 
becomes higher. The stability constraints within the shifted 
unit circle, although useful, require the formulation of sampled- 
data system in the shifted z plane 6 ’ 7 . 

In a recent work on the problem of stability of linear discrete 
systems, or alternately, the necessary and sufficient conditions 
for the roots of a real polynomial to be inside the unit circle, the 
author has succeeded in obtaining a satisfactory criterion 1 . 
This criterion constitutes a major simplification of the Schur- 
Cohn criterion, and yields the final constraints in a usable form 
such that no transformation is required. It is similar to the 
Hurwitz or Lienard-Chipart criteria 8 and has the advantage that 
it may be readily used for the design of linear discrete systems. 

* A portion of this paper has been obtained from the contents of 
the report, ‘On the Roots of a Real Polynomial Inside the Unit 
Circle and a Stability Criterion for Linear Discrete Systems’, by E. 
I. Jury and J. Blanchard, ERL Report No. 425, Issue No. 60, Uni¬ 
versity of Calif., Berkeley; December 26, 1961. 
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ON THE ROOTS OF A REAL POLYNOMIAL INSIDE THE UNIT CIRCLE AND A STABILITY CRITERION FOR LINEAR DISCRETE SYSTEMS 


In this paper an alternate criterion for stability or for obtain¬ 
ing the number of roots inside the unit circle is obtained. This 
criterion is similar to the Routh table used for the continous case 
in obtaining the number of roots in the left half of the s plane. 
It is given in table form and can be calculated by expanding 
only second-order determinants. It has the advantage over the 
preceding criterion in the sense that calculation of higher order 
determinants is no longer required. Furthermore, it is particularly 
useful if the coefficients of the polynomial are given in numbers. 

The derivation of the criterion is based on the theorems of 
Rouche, which were discussed extensively by Marden 4 . The latter 
has obtained a table form of the criterion for the complex 
coefficients and Tsypkin 9 had rearranged this table for the real 
coefficient case. Recently Brown 10 has applied the Marden table 
to the stability test of a fourth order real polynomial. This 
author has obtained a major simplification of this earlier work 
and has shown the connection between the table form and the 
earlier obtained simplified criterion. The application of the 
criterion in the design of linear discrete systems as well as other 
applications constitutes the major discussion of this paper. 


Mathematical Background 

The work of Marden 4 which is based on the earlier work of 
Rouche 11 is first reviewed. Only real polynomials will be con¬ 
sidered in this paper; however, some of the theorems can also 
be applied to the general coefficient case. Let the considered 
polynomial be denoted as F (z) and be written as follows: 

F (z) = a 0 + a t z + a 2 z 2 4-... T* a k z k +... + a n z n 

Consider the sequence of polynomials F Q (z), F x (z)... F$ (z). 
If to each of these polynomials is associated the polynomial 
Fj* (z), whose roots are the reciprocal ones of Fj (z), then 
F j+1 (z) can be obtained from Fj (z) as follows: 

F j+1 (z) = a < b J) F J (z)-a ( n J ljF*(z), j=0, 1, 1 (1) 

where n -j 

Fj( z) — £ a?z k (la) 

k = 0 

It should be noted that all af ] are written as a k . 


F*(z) = z n - J Fj(llz), j = 0,1,.. n -1 (2) 

The following observations, which become evident if, for 
example, Fj (z) is divided by Fj* (z) for; = 0 as defined in (2), 
can be deduced from eqns (1) and (2): 

(a) If F 3 (z) is of degree n-j , then F, +1 is at most of degree 
n-j- 1 . 

(b) The coefficients of F j+1 can be obtained from the 
coefficients of Fj as follows: 


n U+i)^ n U) n U)__ a U) „(./). 
a k — u 0 u k u n -ju n —j— k 


(3) 


(c) F(z) is equivalent to F 0 (z), by the definition of Fj (z) from 
(ia). 

In each polynomial Fj (z), the constant real coefficient 
is denoted by &j, such that 

{d) The zeros of F* (z) are relative to the circle j z | = L 
the inverses of the zeros of F 3 (z). 


The following theorem has been proved by Marden 4 . If for 
the polynomial F(z), p of the products (P k = d 2 .. • 8 k ; 
k = 1, 2, ...,/?) are negative and the remaining n-p are positive, 
then F(z) has p zeros in the unit circle | z | = 1, no zeros on this 
circle and n-p zeros outside this circle. From this theorem it can 
be deduced that the necessary and sufficient condition for 
F (z) — 0 to have all its roots inside the unit circle is 

(5 1 <0,^ 2 >0,...,(5 n >0 (5) 

A convenient way to obtain d k and Fj (z) is naturally to construct 
the following table by noting eqns (2)-(4). 


Table 1. A Stability Procedure Form 



In Table 1, 


a 0 

„(i) 

£*0 ®n-k 

n (2)_ 

a 0 a n -\ 

a n a o 

s a lc — 

\a n a k 

, a 0 — 

M) ^(i) 

a n- 1 a 0 


Furthermore, it should be noted that the elements of row 
2/c4-2 consist of the coefficients of the row 2 k + 1 written in 
reverse order, k — 0, 1, 2,..., n. 

The number of roots inside the unit circle if d k 0, is equal 
to the negative number of the products of 

P k = 5 l 5 2 8 3 ,...,d k , /c = 1,2,3,4,..., n (6) 

where P 1 == <5 X , P 2 = d 1 d 2 ,... 

The necessary and sufficient condition for all the roots to 
be inside the unit circle is when the negative number of the 
products P k is n. This condition is satisfied if and only if the 
following exists: 

(5 1 <0,(5 2 >0,5 3 >0, ...,<5„>0 (7) 


Example 1 

To illustrate the use of the above table, consider the following 
polynomial: 

F(z) = 3-2z-yz 2 + z 3 (8) 
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Polynomial 

Row 

k 

Coefficients 

m 

1 


3 

-2 

3 

~ 2 

1 

F*(z) 

2 


1 

3 

“* 2 

-2 

3 

Fi(*) 

3 

<$i=8 

8 

9 

~~ 2 

5 

~T 



4 


5 

“ 2 

9 

~~ 2 

8 


F,(z) 

5 

rn 1 

cm I 

II 

231 

4 

189 
~~ 4 



Fl(z) 

6 


189 
~ 4 

231 

4 



F,(z) 

7 

($<,££ 1,100 

^ 1,100 





In this case, S x > 0, <5 2 > 0, <5 3 > 0 
therefore 

?!=$!> 0 

P 2~ S 2^> 0 (9) 

P 3 = < 5 1 ^ 2 < ^3 ^ 0 

Since all the Pfs are positive, therefore none of the roots are in¬ 
side the unit circle. The roots of the above equation are z x = 3/2, 
%3 = ± V 2, which verifies that they are all outside the unit 
circle. 


$ cn — 2 co — 3 c/i — 4 

^„_ 2 a. 1 -B,. 1 ) J F(l)-F(-l), n> 2 (14) 

Since 04-x — i^-i) 2 is always positive, and for the roots to be 
inside the unit circle, <5 2 , <3 3 , S n ~ x are all positive; therefore, 
the last condition S n > 0 can be equivalently replaced by: 


<5„>0<s>( —1)”F(1)-F (—1)>0 


If we let a n > 0, then S n > reduces to 15 


F(1)>0,F(-1) 


> 0, n even 
< 0, n odd 


(15) 


(16) 


It should be noted that the last constraint in eqn (7) could be 
replaced by the above equation by letting a n > 0. 

The above simplification may also be utilized to obtain in¬ 
formation on the number P n without having to calculate d n , but 
using instead relationship (14). 

Since the connection between S k and A k = A k 2 — B k 2 is 
obtained, one can apply all the significant simplifications ob¬ 
tained 1 on the A k s and B k s * to d k or to the stability problem 
using this criterion. This connection will be shown in a few 
examples. 

Utilizing eqn (16), one can rewrite Table l in the form of 
Table 2 by noting the following from Tables 1 and 2: 


h - n (1) r ~n (2) A - p - 
u k — **ic — u-k ? a, k — u k ,e k — u k 


W 


Simplification of the Stability Criterion and the Table Form 12 


To simplify the criterion for the number of roots or for the 
stability, the connection between the S k given in this paper and 
Schur-Cohn 2 determinants d fc ’s is established. The determinants 
^I fc ’s are discussed extensively in the literature 1 * 4 and they offer 
an alternate method for obtaining the number of roots inside 
the unit circle. The connection between d k and A k was first 
established and proved by Marden 4 and is given by the following 
relation: 




5 


k -2 


(10) 


This equation could be used to calculate the A k 's from dfs, thus 
avoiding evaluation of higher order determinants. When k = n, 
the above yields 


A =- 


*T 


n-2^n-3^n~4 


or 


? •••5 $n-2 


X _ ?n-2 cn-3 cn-4 f/i-5 c2 c A 

°n~ 0 1 d 2 d 3 ,...,d n _ 3 d tt _ 2 Zf„ 


(U) 

( 12 ) 


and 


§ l = a i 0 1) =b 0 =a 2 0 - a 2 , S 2 = a ( 0 2) =b%-b~_ u 
^3 = a 0 3) = c 0 — c f-2> 

^n-i=a ( o~ 1) = rl-r 2 1 , 

s n =0o - rl) 2 -(r 0 r 1 -r 1 r 2 ) 2 = a% ) 


The necessary and sufficient conditions for the roots of 
F(z) = 0, with a n > 0, to lie inside the unit circle are given as 
follows: 


F 


(1)>0,F( —1) 


>0, rc even 
<0, nodd 


l«ol <a„ 
l*o!>|6.-il 

|c 0 l>|c B - 2 | 

Idol ^ I— 3 j 


(n — 1) constraints 


kol>k 2 l 


(17) 


(18) 


In an earlier work 1 it is proved that 

4=^ 2 -B„ 2 =U„- 1 -B„_ 1 ) 2 F(l)-F(-l) (13)* 
Substituting this expression in (12), the following is obtained 


It is noticed from Table 2 that a certain computation involved 
in the dotted entry in row In - 3 becomes redundant. Therefore 
the correspnding second-order determinant to be calculated in 
entries In — 4 and In — 5 is also redundant. 


* The stability constants A n , B ns A n - V B n ^ x are defined and discuss¬ 
ed in earlier investigations 1 . 


-1 and 2 = 


' B k — A h -. x A lc+1 — B k . 
=:(A n ^B n ^fF(l)FQ 


1 ^/c+l> 


k=* 2, 3, 
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Table 2. A Simplified Stability Procedure Form 


Row 

z° 

z 1 

Z 2 

z 3 

... z n ~ ]i 


z n~ l 

z n 

1 

a 0 

a\ 

"2 

(to 

. •. a n -!c 


a%— i 

an 

2 

On 

an —l 

a n —2 

an —3 

...ax 


ai 

a 0 

3 

bo 

bi 

bo 




bn—i 


4 

b n - 1 

bn— 2 

bn—o 




b 0 


5 

Co 

Ci 

c 2 



Cn— 2 



6 

C/i—2 

Cn— 3 

Cn— 4 



Co 



In—3 

So 

Si 

i s 2 

s 3 





In—A 

So 

s a 

Si 1 

s 0 





2n —3 


: ri l 

r* 







The number of roots inside the unit circle, if none of the 
constraints in (18) are equal, is given by the negative number 
of the product 


1 >0-0834, |b 0 l>l*„-il ( 24 ) 

0-9936 >0-5256, |c 0 |>|c„_ 2 | (25) 

The system is stable and therefore all the roots are inside the unit 
circle. 

An alternative simplified form of the stability constraints 
given in eqn (18) can be obtained by noting that 


b o = al-a*<0 | 

and r 

(26) 

ki>k-ii 1 


These two equations can also be written as 


b 0 ~b n - 1 < 0 , b 0 + b n - 1 <0 

(26 a) 

Similarly, for 

^■0 Z=C 0~~ C n-2 > ^ | 

and j 

(27) 

\d 0 \>\d n -3\ J 

one can write 

d 0 d n — 3 > 0 , d 0 d n —3 > 0 

(28 a) 


Similar relationships could also be obtained for the other con¬ 
straints. The new stability criterion can be summarized as follows: 


n-even* 


P k = [|fl 0 | - kI] [l&ol - k- ill. •••> tko - rl I - \r 0 ri ~ r a r 3 1] 

fc = l,2, ...,n (19) 

with 

P 1 = [|a 0 | - |a„|], P 2 = [kl - kl] [kl ~ k-iG> ( 20 ) 

The difference between n and this number yields the number of 
roots outside the unit circle. 

Example 2 

p ( z ) = 0-0025 + 0-08 2 + 0-4126 z 2 -1-368 z 3 + z 4 , n = 4 (21) 


Table 3. Stability Test for Example 2 


Row 

z° 

z 1 

z 2 

z 3 

z 4 

1 

0-0025 

0*08 

0-4126 

-1*368 

1 

2 

1 

-1-368 

0*4126 

0*08 

0*0025 

3 


1-368 

-0*4116 

-0-0834 


4 

—0*0834 

-0-4116 

1*368 

-1 


5 

0-9936 

. -1*402 

0*5256 




F (1)>0, jP (— 1) >0 


a 0 c0, a,Q-\- ci n >0 

Cq C n — 2^>0, Co”h^ n _2^ > ^ 
e o — e n -4.>0, ^ 0 + e /i~4 > ^ 


> n-constraints 


S o -S 2 >0, 5 o + 5 2 >0 J 

n-odd* 


(28 b) 


F (1)>0, F(-1)<0 


h 0 b n _i<0, 1<° 

d 0 — d n - 3 >0 ? d o + d n - 3 >0 

fo fn — 5 ^ fo^fn-S^® 


> n — 1 -constraints 


r o -r 2 >0> r 0 + r 2 > 0 J 

The above form can be further simplified by noting that each of 
the above constraints is divisible by the preceding constraints 
with alternate signs; i.e. c 0 — c n - 2 is divisible by a 0 + a n , etc - 
This property is utilized in the next section to show the equi¬ 
valence between the table form and the determinant form pre¬ 
viously obtained 1 . This simplification is particularly useful if the 
coefficients of F(z) are given otherwise than in numbers. 


For the stability test or determination of number of roots inside 
the unit circle; there is: 

F (1) = 0* 1271 > 0, F (-1) = 2-703 > 0 (22) 

0-0025<1, \a 0 \<a n (23) 


* It should be noted that the stability constraints for w-even could 
also be written in an alternate form such that the starting constraints 
are b 0 - b n - 1 < 0, and b 0 + b n - 1 < 0. In this manner one can show 
that the final constraint is only s 0 — s 2 > 0. Similarly, for n-odd the 
final constraint is only r 0 — r 2 > 0. These forms could be used 
advantageously in the critical study of stability limit . 
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Relationship between the Table and Determinant Forms 1,23 

In a preceding work a stability criterion in terms of the coef¬ 
ficients of F(z) is obtained in a determinant form. Now the rela¬ 
tionship between the stability criterion introduced in this paper 
and the mentioned criterion will be established. The importance 
of this relationship lies in the fact that in order to obtain the 
former criterion 1 only second-order determinant evaluation is 
required. 

To establish the identity between the A lc + B k of the preced¬ 
ing criterion and the coefficients of the table, the following may 


ct 0 ±a n = A 1 ±B 1 

(29) 

bo + b n _ 1 = A 2 ± B 2 

(30) 


The above is evident by direct examination of the terms of 
A x ± B 1 and A 2 ± B 2 and the corresponding terms of the table. 
Furthermore, by regular division the following relationships 
can be further identified 


The above constraints in terms of /fs and B’s can be written by 
noting from 6 0 A 4 < 0 that b 0 < 0 ; for n = 5, one obtains: 

A 2 — B 2 0, A 2 + B 2 < 0 

(38) 

.4 4 — B 4 . > 0 , A 4 -b J ?4 > 0 

(39) 

F(1)>0, F{- 1)<0 

(40) 

n = 6 : 


In this case the constraints are: 


a 0 -a 6 < 0 , a 0 +n 6 >0 

(41) 

c o -c 4 > 0 , c 0 + c 4 >0 

(42) 

e 0 -e 2 > 0 , e o + e 2 >0 

(43) 

F( 1)>0, F( —1)>0 

(44) 


a 3 -b 3 = c ^-^,a 3 +b 3 = c -^±^i 

&0 Cl n CIq < 2 n 


iQ-ru n 

The above can also be written as 


(31) 


Using the relationship introduced in eqns (29), (31), and (34), 
one obtains 


A + B — c ° — c n-2 
3 - 3 C AiTBJ 


following is obtained 


A 4 . ft — i. d n - 3 

4 - 4 (a 2 0 -af t )(A 2 TB 2 ) 



Ai — 

(45) 

(32) 

A 3 — B 3 > 0 , A 2 -j~ B 3 < 0 

(46) 

various terms, the 

A 5 — B 3 < 0, A $ + B$ > 0 

(47) 


F (1)>0, F(-1)>0 

(48) 


(33) 


Following the above procedure the following general formula 
can be obtained: 

= 

_ lp + in-k+1 __ 

(al-a 2 n ) k -Hbl- b 2 n _ /)* “ V.., k + 3 )(A k _ 2 + B k _ 2 ) 

(34)* 

By imposing the stability constraints on the coefficients of the 
table for / 2 -odd and «-even, respectively, the constraints can be 
readily obtained on the A jc + B k which can be seen to coincide 
with the preceding criterion**. This will be illustrated for n = 5 
and n = 6 , respectively. 

n = 5: 

The stability constraints for this case in terms of the con¬ 
straints in eqn (27b) are: 

b o -b 4 < 0 , fe 0 4 -b 4 <0 ( 35 ) 

^2 0, d 0 + d 2 > 0 (36) 

F( 1 )> 0 , F( — 1 ) <0 ( 37 ) 


A 2 ±B 2 = b 0 ±b A ,A^±B, = 7 - 1 


( a o~~ a l) (A 2 -+B 2 ) 


Noting that 
A 2 4 

and b 0 = a% — a\. 

* This relationship has also been verified without the use of division, 
but by using the induction method 18 . The I's and h's are the appro- 
priate entries in Table 2. 

** By modifying Table 2 , slightly, A k ± B 1c can also be obtained 
directly without determinant expansion 1 . 


Following the above procedure, the relationships for the 
general case can be readily formulated 1 . 


Summary of Stability Constraints for Low-order Systems 
n — 2: n = 3: 

CIq < Cl 2 |tf 0 l< fl 3 

F(1)>0, F(— 1)>0 Oq — a^ < a Q ci 2 — ct ^ci 2 

F (1)>0, jF(— 1)<0 

n — 4: 

Form I (1) a 0 -a 4 <0, a 0 H-a 4 >0 

(2) A 2 — B 2 > 0, A 3 + B 3 < 0 

(3) F(1)>0, F( — 1)>0 

Form II A 2 — B 2 < 0, A 2 -hB 2 < 0, 

where A 2 = a 2 0 -al > B 2 = a 0 a 3 -ci l a 4 
A 3 ~~B 3 >0, 

where ^4 3 — B 3 — al F2a 0 a 2 a 4 J r a 1 a z a 4 _ — a 0 a\ 
^ 0^4 ci 2 ci 4 ciq(X 2 ciqCi 2 — 4 -\~ ci 4 -\- ciqCI ^ci 3 

F(1)>0,F(-1)>0 
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The Case F(z) has no Roots on the Unit Circle, but 8 k+l = 0 

Remarks on Use of Table . , 

, illustrated. 

f k (z) = F k (rz) 

Simplification of the Tables which, for r — 1 + 6 and £ is a sufficiently small positive quan- 

In certain cases the numerical values in the rows. (2/c + 1) tity, has as many zeros in the circle as does F k (z). 
could be very large or very small, so that multiplication or di¬ 
vision by a certain constant 2 is desirable. This can be easily Example 

accomplished without changing the results of the stability test. F (z) = — 2 -F 3 z + 2 z 

For instance, in Table 2 the elements of row (2k + 1) are divided . , , , nn 

(or multipled) by 2; then the elements of row (2k +2) are also In this case j t j s noticed that <5j, = ag - a; - 0 (and F(z) 

divided (or multiplied) by the same factor, and for stability the zerQ on the unit circle); therefore, z in F(z) is replaced by 

ratio of the first column of these rows is evidently independent fz _ ^ _ g ) Zj with e = 01, thus 

of ‘2’. Therefore, for the stability test or for obtaining informa- 9 , 97 ,+ 1 62 z 2 

tion on the roots inside the unit, this modification will not alter f k (z) = -2 + 2-1 z + l-blz 

the results. T _ ,, _ . _ „„„ x = 4 _ n -621 2 > 0 . and 


(or multiplea) oy a \ men me cicu^mo ^ m inis ^ -u . - A * 

divided (or multiplied) by the same factor, and for stability the zer0 Qn the unit circle); therefore, z m F(z) is replaced by 

ratio of the first column of these rows is evidently independent _ (1 - e) z, with e = (M, thus 

of ‘2’. Therefore, for the stability test or for obtaining informa- 7 , 77 ,+ 1 62 z 2 

tion on the roots inside the unit, this modification will not alter f k (z)= -2 + 2-1 z + l-blz 

the results. In the above case ^ = 4 - (1-62) 2 > 0, and 

The Case F(z) has Zeros on the Circle 22 |z| = 1, and no / t (l)>0,/ t (-1)<0, or ^ 2 <0 

Reciprocal Zeros* Therefore P 1 = d 1 > 0, P, = M 2 < 0, one negative sign hence 

In the above case a certain i* the unit! It’should^notedThaTthe’same results are obtained 

< VV ». * « * s EO.sia.Te, ,0 0. negative, i.e. 

but F lc+1 = 0, then F(z) has n - k zeros on the unit circle 

\z\ = 1 at the zeros of F k (z). It also has p zeros in the unit Roots Inside a Circle of Radius a < 1 

circle, where p is the number of negative P h 1 < J < k 9 ana 

a = k — p zeros outside the circle. To obtain information on the number of roots insi e a circ e 

of radius <r<l, one may apply a certain transformation on F(z) 
such as the circle of radius a, becomes the unit circle. Hence, the 

Example 3 proceeding method is readily applicable. 

Let F(z) — 1 4- 2z + z 2 + 2z 3 The above problem is useful in the relative stability test of 

Let F(z) — F2z + z + ^ characteristic polynomial. The constraints for F(z) = 

Z 0 Z 1 Z 3 £ Ck . z v = 0, to have all its roots inside |z| = a < 1 will be 

obtained by replacing in the constraints found for <r = 1 the 
F(z) 12 12 following: 

F * (z) 2 12 1 a 0 ^a 0 ,a 1 -+a i <T,-..,a p ^ay,-., a n^w" 

Uz) _ 3 o -3 For instance, those constraints for n = 2 and n = 3 are given 

as follows: 

P*(z) —3 0 —3 n = 2: F= CIQ +Gx z + a 2 z2 > 

p&) 0 0 _ |r3 0 | < a 2 cr 2 

F t (a) = a 0 + a l <r + a 2 <r 1 >0 

It is noticed from the above that 8 k < 0, F 2 = 0, and no (_- a )- a - a, o + a 2 <t 2 > 0 

reciprocal zeros of F a (z); thus one root is inside the unit circle 

and two roots are on the unit circle. The location of these two _ 2 3 0 

obtained as follows: n = 3: F(z)=a 0 + a l z + a 2 z +a 2 z ,a 3 >u 


Example 3 

Let F(z) = 1 + 2z + z 2 + 2z 3 



z° 

z l 

z 2 z 3 

F(z) 

1 

2 

1 2 

F*(z) 

2 

1 

2 1 

F,(z) 

-3 

0 

-3 

Fi*(z) 

-3 

0 

-3 

F % (z) 

0 

0 



It is noticed from the above that <5j < 0 , F 2 — 0, and no 
reciprocal zeros of F a (z); thus one root is inside the unit circle 
and two roots are on the unit circle. The location of these two 
roots is obtained as follows: 

F x (z)== —3 — 3 z 2 == - 3(1 + z 2 ) = 0 
therefore the roots are at z 2i3 = ±7* 

* The case when F(z) has both zeros on the unit circle and 
reciprocal zeros could be treated by a certain modification of this 
theorem 22 . 


|a 0 l< fl 3^ 

\al-a 6 al\>\(a 0 a 2 -<z 2 a l a 3 )\a 2 

F 1 (a) = a 0 + a + a 2 a 2 + a 3 cr 3 >• 0 

p 1 (-a) = a 0 -a i a + a 2 a 2 -a 3 o 2 <0, a<] 
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As an extension of the above theorem, one can also obtain 
the stability conditions or the number of roots of a shifted unit 
circle. This is based on a theorem by Marden 4 given as follows. 

To obtain the number of roots in the circle \z~ s\< 1, 
replace the af s of F(z) by the following: 

ap->tc(k,p)s k ~ p a k 

k-p 

where the binomial C(k,p) = kl/pl (k — p)l 

In particular, the number of roots in the unit shifted circle 6 
is obtained as shown in Figure 1, by letting s = 1 in the above 
shifting transformation. 



Figure 1. A shifted unit circle 


Number of Real Roots of F(z) between - 1 and +1 in the 2 plane 

The above information is useful in locating the roots of the 
F(z) polynomial. This also has an effect on the synthesis of linear 
discrete systems when F(z) is stable. The procedure is based on 
‘Sturm’s sequences’ 8 . It is indicated in the following steps. 

(1) Consider F x (z), the derivative of F(z) 

(2) Divide F 1 (z)/F(z) 9 to obtain a remainder — F 2 (z) 

(3) Divide FJF 2 to obtain a remainder — F B 

(4) Continue the division process until the remainder be¬ 
comes a constant — F n 

(5) Consider the following two sequences: 


The method which has been introduced herein is particularly 
useful if the coefficients of F(z) are given in numbers. It is also 
useful for design if one or two parameters are variable. An 
illustrative example is discussed in detail in Appendix II. 

An alternate form of the table, which is based on a division 
method, is introduced in Appendix I. Both the table and the 
division method could easily be simulated on a digital computer 
for stability study of the characteristic polynomials. 

The connection between the table and division method and 
the earlier introduced criteria of stability is established in this 
paper. The determinant method of the first criterion 1 is particu¬ 
larly useful for design, as is also shown in Appendix II. Further¬ 
more, the evaluation of those higher-order determinants in the 
first criterion can be easily deduced from the table form which 
requires evaluation of only second-order determinants. 

It is hoped that the earlier introduced criterion 1 , in combi¬ 
nation with the discussion included herein, will finally solve the 
stability problem in a concise and satisfactory form. The future 
application by workers in this field will undoubtedly indicate the 
usefulness of these three stability criteria. 


Appendix I — A Stability Test Using a Simple Division 14 

In this Appendix an alternate form of the stability test given 
in Table 2 is introduced. It is based on simple division of poly¬ 
nomials, and its derivation could be obtained from Table 2 or 
by using Rouche’s Theorem directly. Since Table 2 is derived, it 
is used for obtaining the simple division method. 

Stability Test 

A necessary and sufficient condition for the real polynomial 
F(z) = a 0 + a 1 z + a 2 z 2 -f ... + a k z k H-... + a n z f \ with a n >0 

(49) 

to have all its roots inside the unit circle in the z plane is 
obtained as follows: 


F(-l) Fi(-l) F 2 (— 1) F n ^ 1 (— 1) F ni to obtain x changes 
of sign, and F(l) F^l) F 2 (l) F n . t (1) F n , to obtain N +1 
changes of sign. 

The number of real roots V of F(z) between (— 1) and (+1) 
is given by 

e== N -i- N +i 

The general conditions for the roots of F(z) to be real can be 
obtained using the early work of Hemite and the more recent 
work of Romanov 13 . In combination with the stability con¬ 
straints one can obtain the necessary and sufficient condition for 
the roots of F(z) to be real and to lie between (— 1) and (+1), 
or between the origin and (+ 1) in the unit circle 19 . 

Conclusions 

It has been shown in this paper that one can easily obtain 
information on the number of roots of a real polynomial in the 
z plane. The method illustrated is based on the use of a table 
form, which consists of two different forms, Table 2 being the 
simpler form. A necessary and sufficient condition for all the 
roots to lie inside the unit circle (or equivalently, for the linear 
discrete system to be stable) may be obtained from the table. 


F* (z) = z"F(l/z) 

= a n +a„_ 1 z+a n . 2 z 2 + ... + a n _ k z k +...+a 0 z n (50) 

Obtain the stability constraints a k by simple division as shown: 


Hz) 
F* (z) 


«o + 


-Fi(z) 

FUz) 


a o "b + 


FAz) 

F*( z ) 


a 0 + a 1 +a 2 


+ 


H 0) 

Ft(z) 


... -*a 0 + cti + a 2 +... + oc n _ 2 


, -F«-i(z) 

Fn-l( z ) 


( 51 ) 


where F x (z), F 2 (z), ... are the remainder polynomials obtained 
from the division and F* (z),F* 00 • • • are their inverses, as defined 
in (50). To obtain the inverse of F^z), one factors out the 
common factor ‘z’ or its powers before applying (50). Similarly 
for Fo(z) and F 3 (z),... 

Satisfy the following constraints for the stability test: 


F ( 1 )> 


0,^-1) 


> 0, n even 
< 0 ,n odd 


KI<1, k=0, 1,2,...,n —2 


(52) 
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The proof of the test lies in the use of the preceding stability 
criterion, for it can be easily established that the identity of oc^s 
and the stability constants a 0 , b 0 , c 0 ,... of Table 2 are as follows. 


oc 2 = 


C n-2 

c 0 


a 


n 


-2 “ 


r o 


From the stability constraints imposed previously on the 
a\ b’ s, ..the constraints to be imposed on the oc k 's are readily 
established as explained in (18). 

The number of roots of F(z) inside the unit circle is equal to 
the number p which is equal to the negative products P k de¬ 
fined as: 






In this case 

a 4 =l,a 3 =-1-368, a 2 =0-368+0-2223 k, 
a t = 0-397 k, a 0 = 0-0123 k 

From Form II of the stability constraints for the fourth- 
order case, 

A 2 + B 2 <0, 0-000151 k 2 -0-4142 k -1 <0, -2-4< k <2740 
A 2 -B 2 <0, 0-000151 k 2 + 0-4142 k-l, -2740</c<2-4 
A 3 -B 3 > 0, -(fc--0-72) (k+ 5-5) (k+ 247)>0, k< -247, 

or — 5-5<fc<0-72 

F(1)>0, fc>0 
F (—1)>0, fe< 16-75 

From the above constraints on k it is evident that k maK = 0-72. 
From Form I, 

\A t \<B u 10-0123 fc|<l,fe<81 

A 3 -B 3 >0,k<-247, or -5-5</c<0-72 

A 3 + B 3 < 0, - (k - 6-27) (k - 279-55) (7c + 1-37) < 0, 

— l-37<fc<6-27, or /c>279-55 


fc = l,2,3, 


with 


Pi = [l«ol-l],-Pa = [l«ol-l] 


and \ oc n ~ x | is obtained from, 



F.-i(z) 

Ft i(z) 



n 


Appendix II—Fourth Order Example 115 ’ 17 

In the sampled-data system shown in Figure 2 , the use of the 
stability constraints discussed for the fourth-order case to obtain 
the maximum value of the gain Td for the stability is shown. 



Figure 2. A sampled-data system with pure delay 


The System Transfer Function 


where 


C*(z) G?(z) 

R*(z) 1 + G?(z) 


<??(*) 


0-2223 


(z+0-03) (z +1-755) 
^z 2 (z-l)(z-0-368) 


The characteristic polynomial 1 -H G*(z) = 0, can be written 
as follows 17 : 

z 4 -1 -368 z 3 + (0-368+0-2223 k) z 2 + 0-3974 kz + 0-0123 k = 0 


F(\)>0, k> 0 
F(- 1) > 0, k < 16-75 

From the above constraints again, as expected, /: max = 0-72. 

It may be noted that the critical equation which gives the limiting 
gain in this case is A 3 — B 3 > 0. Thus in this case only the 
equations A 3 — B s = 0 and F(l) = 0, F(— 1) = 0 need be 
solved. Similarly, for higher-order systems which are initially 
stable and for which some parameters change so as to approach 
instability, only the equations i = and F 0~) = 

i) = o are the most critical ones, which need be solved 1 ’ 20 . 

This research was supported by the United States Air Force 
Office of Scientific Research of the Air Research and Development 
Command , under Contract No. AF\^> (600)—1521. The author is 
grateful to Mr. Jean Blanchard for his co-operation in the early 
stages of this work. 
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DISCUSSION 


M. Hamza, ETH Zurich , Switzerland 

Determining the stability of feedback control systems is a problem of 
major importance, consequently it is very useful to have efficient 
methods for checking system stability. This work is a definite contri¬ 
bution in that direction; it is lucidly written and the stability criteria 
presented are simple and practicable. 

Professor Jury considers real polynomials and mentions that some 
of the theorems he gives can be applied to the general coefficient case. 
Could Professor Jury please indicate which theorems he is referring to ? 
Further, if the coefficients are of the general type which methods for 
investigating system stability does he recommend? Does the method 
recommended cover systems having a pure delay? 


E. I. Jury, in reply 


It may be indicated that the theorem quoted after eqn (4) is applicable 
to the general coefficient case. However, in this case eqn (3) should be 
modified to read 


a k —a 0 a k —a n „ja n -j 


where die is the complex conjugate of ajc. 

Table 1 is also applicable to stability tests for the general coefficients 
case, provided the entries of the second, fourth, ... rows are the 
complex conjugate of the first, third, ... rows. If the pure delay in 
the system is approximated by rational function of z, then the stability 
criterion introduced is equally applicable. 


major preoccupation. My main results have been a new method of 
deriving with the singular cases. In addition, I have used similar 
techniques to prove the Routh and Hurwitz criteria and their various 
extensions. These proofs are shorter and simpler than any others I 
have come across and I hope to publish them shortly. 

When the characteristic equation of a continuous or discrete linear 
system is given the discussion of its stability can take two forms. First, 
if numerical values are given for all the coefficients the simplest proce¬ 
dure with a continuous system is, in general, to use Routh’s method. 
The corresponding process with a discrete system is that described in 
Table 1 of the paper, or the equivalent and slightly simpler process 
given in Appendix I. If, however, it is desired to consider the effect of 
varying one or more of the coefficients or to carry out a general 
theoretical investigation, it is usually better with a continuous system 
to use the determinant sequence of Hurwitz. The equivalent determi¬ 
nants for a discrete system are those which Professor Jury denotes by 
Aic dt Ejc, but which, unfortunately, he has been unable to quote in 
this paper. 

E. I. Jury, in reply 

Because of space limitation, I was unable to discuss in detail the 
formulation of Aic dz Bk. However, these are discussed in detail in 
References 1 and 23 of the paper. One of the major objectives of this 
paper is to establish the connection between the table form, the 
division method and the determinant method. I look forward to 
reading Professor Brown’s contribution on this general problem. 


B. M. Brown, Royal Naval College, Greenwich, London S.E. JO, 

The development of linear discrete systems and digital processes in the 
last 20 years, has led to an increase in importance of the corresponding 
stability theory. This theory was initiated some 40 years ago by Schur, 
Cohn and others, but until recently the only easily available account of 
it seems to have been in Marden’s monograph. This is concerned 
primarily with polynomials with complex coefficients and is therefore 
more complicated than is necessary for a discussion of the practical 
problems of stability. Marden made certain original contributions, 
but since then not much work seems to have been done on the problem 
which, in view of its fundamental nature, is rather surprising. Professor 
Jury is therefore to be commended for undertaking his detailed and 
exhaustive study. Some months ago he was good enough to send me 
some of his work, since when this fascinating problem has been my 


P. M. Parks, Department of Aeronautics, The University , Southampton 

Given the polynomial 

P(z) = z n +a 1 z n ~ 1 +a 2 z”~ 2 + ... + a n 

the Schur-Cohn criterion states that all the zeros of P(z) = 0 lie 
within the unit circle |z| = 1 if, and only if, the principal minors of 
the matrix B 

min (i, j ) 

Bij ~ Y ( a i~p a j~p~ a n + p-i a n + p-j) 

P = 0 ■ 

where a 0 = 1 

are all positive, that is x'B X is a positive-definite quadratic form. 
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Consider the step-by-step system in a state space 

x r A x r _ i 

where A is the matrix 


It would also be interesting to compare for numerical polynomials 
the labour involved in calculating the B ij , Table 2 and the determinant 
ji n _ x =p B n -v For design purposes, the algebraic inequalities I obtained 
using the determinant method seems to be the simplest form. 


A — 


0 1 
0 0 


0 0...0 

1 0...0 


0 0 


-a„-a n _ i 


0 


- a n _ 


n — 2' 


0...1 




The vector sequence X x , X 2 , ..., tends to the origin if, and only if, the 
latent roots of A are all within the unit circle. 

Consider now the tentative Liapunov function 


V r = x r Bx r 

where B is the Schur-Cohn matrix above. 

Then 

Vf. — V r — i = x r Bx r x r _ iBx r — i 


M. Mansour, Inst it ut fiir Automatik und Industrielle Elektronik, 
Altbachstr. 531 Dietlikon\ZH, Switzerland 


It is worthwhile studying the system for instability before applying 
stability criteria, because sufficient conditions for instability are, in 
general, easier to apply. Therefore there is no need to carry out the 
calculations of the stability criteria if the system instability can be 
easily detected. 

Obvious instability sufficient conditions (at least one root of 
F(Z ) = 0 is outside the unit circle) are, for example, 


Ft 0 ) 


>1 


F{ 1)>2" 
a n <0 


■F (— 1) > 2 n I 
a „ <0 J 


n even 


F(2)>3” 
a„ <1 


F (—2) >3"I 
a n <1 J 


n even 


and so on 


= x'^(A'BA-B)x r ^ 

Now it turns out that 

x r ^ 1 (A'BA-B)x r - 1 

is negative semi-definite, in fact, minus one times a perfect square: 
for example, when n = 4 


n odd 

Applying this to criteria (17) and (18) in the paper, there may be no 
need to apply (18) if F(\) > 2»a„, similarly for (27b). 

It is noted that the above instability conditions represent limits of 
ranges the outside of which is definitely unstable. 


B = 

1 — a\ a 1 -a 3 a 4 a 2 -a 2 a A a 3 -a 1 a 4 

l + a\ — a\ — a\ a l + a l a 2 -a 2 a 3 — a 3 a 4 a 2 — a 2 a 4 

symmetrical \-\-a\ — a\ — a\ a 2 — a 3 a 4 

a — a\ 

and 

V r -V^ x =-{(\-al)x r - ul Hai-asadXr-u 2 

~\-a 2 (1 — ^ 4 ) 1,3 F (^3 ^ 1 ^ 4 ) x r - 1 , 4 } 

Hence necessary and sufficient stability conditions follow from the 
positive-definiteness of B, provided V r - K r _ x is not identically zero 
for any sequence of vectors. 

Now consider the row vector 6 f where 

<y' = [l — al, a 1 —a 3 a^ a 2 (l — ^ 4 )? ^3~ a x a ^~\ 

if there exists an x* not zero such that X*, Ax*, A*x* and A 3 X* all 
lie in the hyperplane - 0, then the determinant of the matrix D 
formed of the rows tf', &A, tf'A 2 and <?'A 3 must vanish, as Vx*_~= 0. 
By row operations on D which make it a symmetrical matrix D* it 
turns out that 

D*nB and detD = detD* = detB 

and so if B is positive definite det D 4= 0 and the sequence X*, Ax*, 
A 2 X* and y4 3 X* cannot all lie on the hyperplane ff'x = 0 and so 

y r _ V r -i cannot be identically zero. This completes the proof of 

the Schur-Cohn criterion using Liapunov’s second method. 


E. I. Jury, in reply 

I was very interested to read Dr. Parks’ useful contribution. It would 
be interesting if, in the future contribution, he might obtain my 
stability constraints from Liapunov’s second method. My constraints 
are much simpler than Schur-Cohn for the real coefficients case and 
their number is half of that of Schur-Cohn. 


E. 1. Jury, in reply 

Mr. Mansour, in his contribution, has indicated some necessary 
conditions for instability which can be used for preliminary checking 
of numerical polynomials for stability. Some of these conditions are 
indicated in eqns (17) and (18) of the paper. There exist several other 
conditions which are discussed in detail in References 2 and 22. 
The basis of Mr. Mansour’s inequalities can be easily obtained 
from the upper limit of the product of the roots. 

D. Siljak, Electrotechnical Faculty, University of Beograd, Beograd, 
Macvanska 8, Yugoslavia 

The paper represents a useful contribution to the investigation of 
absolute stability of the linear sampled-data control systems. By the 
proposed numerical procedure, it is also possible to determine the 
variable coefficients of the system characteristic equation so that 
all the roots are located inside the unit circle in the z plane. 

However, in the application of the aforementioned system, it is 
often necessary to evaluate the values of the variable coefficients so 
as to have the system with the prescribed degree of relative stability. 
Then, the attention should be focused on the relative damping region 
in the s plane as shown in Figure A. 
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By the application of Mitrovic’s method 1 ’ 2 , it is convenient to 
map the damping region of Figure A into the plane of any of variable 
coefficients, say a v and a q (n>p>q>Q), of the characteristic 
equation 

F (z) = a„z n + a„ _ 1 z n ~ 1 +... + a p z p +... + a q z q 

+ ... + a i z + a 0 = 0 (1) 

By considering the coefficients a p and a q as variables g and tj, 
respectively, and using the relationship 


rr -~C^) T 


( 2 ) 


where £ is the relative damping coefficient, co n is the undamped 
natural frequency and T is the sampling period, one obtains the 
following equations 2 


E a*e 

k = 0 

£ _ 


-(k-p) COn(T 


<^_ p (-C0Sffl„7yi-£ 2 ) 


^-pC-cos^r^/i-c 2 ) 


rj = 


E a k & 

k = 0 
k*q 


“( k~q ) a> n $T 


^_ p (-cosco„ryi-C 2 ) 


— cosa)„T —C 2 ) 


( 3 ) 


In the Ogp plane, the eqns (3) represent the loci of points cor¬ 
responding to the roots with constant damping coefficient £. If the 
region shown in Figure A is mapped into the Ogp plane, using the 
eqns (1), (2) and (3), it is possible to determine all values of the 
variables £ and r\ which correspond to the degree of relative stability 
indicated by M in Figure A. 
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E. I. Jury, in reply 

I might indicate that on page 147, under the heading ‘Number 
of Roots Inside a Circle of Radius a < 1’, one can use these 
results to obtain analytic conditions on relative stability of a different 
form than the one proposed. From the form indicated in Figure A 
of Dr. Siljak’s discussion, one can obtain analytic conditions, but 
these become quite complicated. In this case the graphical procedure 
indicated by Dr. Siljak becomes more convenient, although quite 
involved graphically. 


M. Thoma, Institut fur Rege lungs technik, Technische Hochschule 
Darmstadt, Darmstadt, Heidelberger Str. 11, West Germany 

In his interesting paper, Professor Jury deals mainly with the problem 
of testing the stability of linear sampled-data systems. The advantage 
of the given reduction method is that the bi-linear transformation, 
as the solution of determinants of higher than second order, is also 
avoided. But the theorem of Marden, on the right-hand side of 
page 43, and therefore eqn (7), is only valid if none of the 8 k 
' 7" * • -> n ) is missing. However, this case may easily occur with 

stablejystems because it is not necessary that all the coefficients 
encKk ~\,2 3 n) of the equation F(z) = 0 exist. Therefore, the 

statement (a) on page 143 is not quite general. It must be read: If 

1S of degree n —j then F j+1 is at highest degree of n—j— 1 
For example, if J 


F(z) = a 0 + a 2 z 2 + a 5 z 5 

then 

F t (z)= F a\ — al'F a 2 a 0 z 2 — a 2 a 5 z 2 

which is of degree three and <5 2 does not exist. 

In my paper I gave almost simultaneously, but independent of 
Dr. Jury, another derivation of this result. It is more direct and there 
can be found a somewhat more uniform consideration of the problem. 
If we define eqn (1), not as it is done in the paper by Dr. Jury, but in 
the following manner: 

F, + 1 (z)=l[^,F y (z)-a^(z)], y = 0,l,. 


then the constant term of the polynomial in the squared brackets 
is zero. However, as proved by Cohn 2 and Thoma 1 , the polynomials 
Fj(z) = 0 and z • F,+i(z) = 0 do have for 


l a n-/l>l a 0 ) | 

the same number of roots inside the unit circle. Because the equation 
z • F j+1 (z) = 0 has the roots z = 0 in the unit circle we have, for stable 
systems, just to show that the polynomial F j+1 (z) of degree n —j— 1 
has only roots with | z| < 1. Thereby eqn (3) changes to 


ai j ^lj_ k = ai J lj ■ a[ J lj_ k — a <P f b = 


(k=0, —1— j) 


y'+l is the number of reductions (e.g. for j = 0 we get the n— 1 
coefficients of the first reduction, etc.). But all the roots of the poly¬ 
nomial can then only lie in the unit circle if the absolute value of the 
coefficient of the term with the highest power a n (highest coefficient) 
is larger than that of the constant term a 0 (last coefficient), i.e. 


I^nl |ttol 

It is easily shown that for 

kliSKI 

not all the roots can lie inside the unit circle. We therefore devide 
the equation 


F(z) = a 0 + a 1 z+...+a n _ 1 z"“ 1 +a„z" = 0 

by a n ; it yields 

^>i 

a n 

because we assumed that 

I^hI ^ \a 0 \ 

We split this equation into the factors 


F(z)=(z-y„) (z—y„_i)... (z- 7l )=o 

where n are the n roots of F(z). After multiplying out the last equation, 
the coefficient of z n is equal to 1 and that of the constant term to 

( l)°7n * 7«—l • • ■ 71 

For 

. . Kl^ol 

it is necessary that 

77 =lT„-y„- 1 ...y 1 |>l 

u n 

However, this is just possible if either all | y t | = 1 or at least one 
root has an absolute value greater than 1. For a stable system it is 
therefore necessary that in the given equation F(z) = 0, as in all 
reduced equations, Ffz) = 0 the absolute value of the highest 
coefficients is in contrast to eqn (18) larger than that of the last 
coefficient 

l<#bl>l<# ) l, 7—0,1, ...,n —1 
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Besides this, the description 1 given can easily be extended to determine 
the number of roots inside, outside, and on the unit circle, and to 
polynomials with complex coefficients which is important for the 
determination of the root distribution 3 . As we discovered, the given 
procedures 1 * 3 are also very suitable for the automatic calculation of 
higher order polynomials with a digital computer. Further study 
of this problem can be found in the discussions 4 * 5 . 

References 
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E. I. Jury, in reply 

(1) The theorem of Marden, quoted after eqn (4), is quite correct. 
In some cases when certain coefficients of the polynomial F(z) do not 
appear, then some of the djc. do not exist. For such a special case the 
modification of Dr. Thoma is quite correct. This modification has 
already been incorporated in the page-proofs of the paper. These 
special cases are discussed in detail in References 2 and 22. 

(2) In my earlier contribution 12 * 14 which slightly preceded Dr. 
Thoma’s contribution, I have briefly presented the table and division 
method. In this paper the proofs and extension are indicated. In my 
Table 2 the stability test is simpler than Dr. Thoma’s method because 
of the use of the auxiliary constraints F( 1) > 0 and (— \) n F(— 1) > 0 
as indicated in eqn (17). This fact and other points are discussed with 
Dr. Thoma in References 4 and 5 of his discussion. Furthermore, 
if the first constraint in eqn (18) is written as a n > \a 0 \, then our 
first (« — 1) constraints become identical. 

(3) In the present contribution, I have also discussed the root 
distribution within the unit circle as well as the stability condition. 
The singular cases are briefly discussed, and in Reference 22 a detailed 
study will be available. Finally, in my discussions with Dr. Thoma’s 
contribution much light has been shed on the solution of this inter¬ 
esting problem, and the reader’s attention has been drawn to the 
contribution of European and American authors in the past. 


Ya. Z. Tsypkin, Institute of Automation and Telemechanics, Moscow 
1-53, Kalanchevskaya 15-A, U.S.S.R. 

Professor Jury’s paper gives an exhaustive solution to problems of 
determining the stability and the degree of stability of sampled-data 
systems, and it is difficult to add anything to the results obtained. 
I should merely like to make a few comments on this field of questions. 

Algebraic criteria of stability are obviously suitable for character¬ 
istic equations of low powers, since the coefficients of these equations 
are linked by complex relationships to the parameters of the sampled- 
data system. 

For high-order sampled-data systems we can speak of determining 
the stability of the system according to numerically-specified para¬ 
meters. 

The simplest way to determine stability seems to be by the tabular 
form 1 , corresponding to Schur’s criterion, or the modifications of it 
given in Jury’s paper. It would be desirable to compare all the numer¬ 
ical methods of stability determination from the point of view of the 
amount of computing operations. 

The use of the Schur theorem for determining the number of real 
roots, lying on the segment — 1, + 1, involves cumbersome computa¬ 
tions. They would be justified if such distribution of the roots had a 
particular purpose. I should like to ask the author when the need 
arises for such distribution of the roots. 

Reference 

1 Tsypkin, Ya. Z. Theory of Linear Sampled-data Systems, 
Moscow, 1963 


E. I. Jury, in reply 

The discussion on the number of real roots of F(z) between — 1 and 
+ 1 in the z plane is mainly of academic interest. However, the 
conditions on the roots to lie between (0,1) in the unit circle is of 
practical significance. It is related to the problem of aperiodicity or, 
alternately, for the sampled-data system response type of finite 
oscillations. This type of system is used in instrumentation feedback 
servos where excessive oscillations are not desirable. A thorough 
discussion of this aperiodicity problem is indicated in Reference 19 
of the paper. As for comparison of the various methods for numerical 
test of stability, my investigation has shown that both the table form 
and the determinant method (provided the latter is written in the 
proper form) are equivalent 1 . However, for hand computation, the 
table method is more convenient. 

Reference 

1 Jury, E. I. On the evaluation of the stability determinants in 
linear discrete systems. Inst. Radio Engrs. N.Y. P.G.A.C. (July 1962) 
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Summary 

The analysis of non-linear oscillations in sampled-data control 
systems is treated. The system considered here has the non-linear 
element in discrete signal portions. In the first half of the paper the 
strict analytical method is proposed. By using it, the periodic equi¬ 
librium states and their stability can be strictly analysed, without any 
assumption on the waveform, if the period of the oscillation is an in¬ 
teger multiple of the sampling period. 

In the second half, two different applications of describing function 
methods to the sampled-data systems are proposed. One is defined in 
the same way as in continuous systems, and it is accurately applicable 
to the sinusoidal oscillation of which the period is sufficiently larger 
than the sampling period. The other is defined by considering the phase 
relation of oscillations at sampling instants, and it is applicable when 
both frequencies are closely approached. 

Moreover, in some examples illustrated, the existence of ‘hard 
self-excitations’ in sampled-data systems is suggested. 

Sommaire 

L’auteur expose l’analyse des oscillations non-lineaires dans des sys¬ 
temes de commande du type a donnees echantillonnees. Dans le sys- 
teme considere ici, l’element non-lineaire reside dans le signal qui est 
echantillonne par portions discretes. La premiere partie de la commu¬ 
nication traite de la methode strictement analytique. Cette methode 
permet d’analyser strictement, sans aucune hypothese sur la forme des 
oscillations, les etats d’equilibre periodiques et leur stabilite, cela pour 
autant que la periode des oscillations soit un multiple entier de la 
periode d’echantillonnage. 

Dans la seconde partie, on suggere deux applications diflerentes 
des methodes utilisant une fonction descriptive a des systemes a 
donnees echantillonnees. Pour Tune, la methode est utilisee de la 
meme maniere que dans un systeme continu: elle s’applique de fagon 
rigoureuse si Foscillation est sinuso'idale et a une periode suffisamment 
grande par rapport a la periode d’echantillonnage. Pour l’autre, il 
faut tenir compte du dephasage entre les oscillations et l’echantillon- 
nage: elle s’applique lorsque ces deux frequences sont voisines. 

En outre, l’existence possible d’‘auto-excitations dures’ (hard self¬ 
excitations) dans les systemes a donnees echantillonnees sera signalee 
a propos de quelques-uns des exemples traites. 


Schwingungen in den Abtastaugenblicken; sie ist dann anwendbar, 
wenn beide Frequenzen nahe beieinander liegen. 

Einige Beispiele lassen zudem vermuten, daft auch in Abtast- 
systemen „harte“ selbsterregte Schwingungen auftreten. 


Introduction 

The analysis of a sampled-data control system with non-lineari¬ 
ties is fairly difficult due to the presence of discontinuities of 
the signal in addition to the non-linear elements, such as 
saturation, backlash, relay, etc. However, the same methods as 
for continuous systems may be applicable to sampled-data 
systems which contain a non-linear element in continuous 
signal portions, if they are slightly modified, for example, by 
introducing the z transformation. Thus, a peculiarity of the 
analysis for non-linear sampled-data systems occurs when non¬ 
linearity exists in the discrete signal portion, because the signals 
in this portion contain many higher harmonic frequency com¬ 
ponents. This difficulty is more pronounced when the frequency 
of the non-linear oscillation approaches the sampling fre¬ 
quency. 

From this viewpoint, such systems as indicated in Figure 1 {a) 
and ( b ) are considered in this paper. If the hold circuit H (s) is 
a zero-order hold, the phenomena observed in the circuits ( a ) 
and ( b ) are evidently identical. 

Controller Nonlinear Zero order Controlled 
element hold system 



Figure 1. Non-linear sampled-data control system 


Zusammenfassung 

Der Aufsatz behandelt die Untersuchung nichtlinearer Schwingungen 
in Abtastregelsystemen. In dem hier betrachteten System tritt die 
Nichtlinearitat wahrend der diskreten Signalschnitte auf. Der erste 
Teil des Beitrages behandelt die strenge analytische Methode. Mit 
ihrer Hilfe lassen sich die periodischen Gleichgewichtszustande und 
ihre Stabilitat genau untersuchen, wobei Annahmen fiber die Wellen- 
form nicht notwendig sind, wenn nur die Schwingungsperiode ein 
ganzes Vielfaches der Abtastperiode ist. 

Im zweiten Teil werden zwei verschiedene Anwendungen der Be- 
schreibungsfunktionen auf Abtastsysteme vorgeschlagen. Eine ist in 
gleicher Weise wie bei den kontinuierlich arbeitenden Systemen de- 
finiert und kann mit guter Genauigkeit auf sinusformige Schwingungen 
mit einer gegenfiber der Abtastperiode genfigend groBen Perioden- 
dauer angewendet werden. Die andere betrachtet die Phasenlage der 


Analysis of Non-linear Oscillation in a Sampled-data Control 
System by the Method Without any Approximation 

If both the input and the output of a non-linear element are 
expressed in the form of pulse series x* (/) and y* (/), respec¬ 
tively, in Figure 2 the following relations hold: 

x*(t)= y X;<5 (t-iT) 

i= ° ( 1 ) 

00 CO 

y*(t)= Z y t S(t-iT)= Z aiXi§(t — iT) 

i = 0 i - 0 

where oci is a weighting coefficient for the input pulses and 
varies with the characteristics of the non-linear element and the 
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Figure 2. Input and output of non-linear element 


From eqns (2)-(4), for the equilibrium state of period / T, the 
following relations are obtained; 

(B 1 + A l a,)x l + (B 2 + A 2 cc l - i )x l - 1 + .+(B, + ^4,a 1 )x 1 =0 

(B 2 + A 2 a l )x l +(B 3 + A 3 a l - 1 )x l - l + ... +(£ 1 +4iai)xi=0 


magnitude of each value of x # . When the input x* (/) is a periodic 

pulse series x*t of period / T, where I is an integer, then the 
/ kinds of <x, (/ == 1 — l) are defined corresponding to 1 kinds 
of input pulses x v x 2 , ... x b respectively. By applying this 
consideration to the non-linear sampled-data system indicated 
in Figure 3 ( a ), it can be easily found that the equilibrium state 
for the periodic oscillation of period l T, can be treated on 
the equivalent circuit shown in Figure 3 ( b ), where the rotary 
switch S is successively connected with the elements tx { in the 
order of «j, « a , ..., * t , ••• with the time interval T. x t 

denotes the value of the error signal at the instant when ot t is 
connected. 



(b) 

Figure 3. Sampled-data system: ( a ) original system; (6) equivalent system 


Equilibrium States 

The z transforms of x*(t) and y*(0 are represented by 


1 


rv; rys v—i •'v . 

.**(0= E 7: -K 

i= 1 (1 — 2 ) 


( 2 ) 


Assuming the system to be in self-oscillation and the steady- 
state value of the system input r (/) to be zero, gives the lelation 


X*(z)=-W*(z)=-G*(z)Y*(z) (3) 


where W* (z) is the z transform of the system output w it) and 
q* (z) is the pulse transfer function of the linear element 
H(s) G (.?). In general, G* (z) is represented by 


G*(z) = 


q 0 + a]Z 1 +... + a„,z 
1 F b ±z z 


(4) 


(B, +A l u l )x l + (B l + A 1 cc l -. 1 )x l . l + ... +(B l - 1 +A t - 1 o(. 1 )x i 

= 0 (5) 

where B, and A t (/ = 1 ~ /) arc 

B l = l + b l + b 2 i +.... A l = a 0 + a, + a 2l +.... 

B 2 = b 1 +b l+1 + b 2 i + i +.... A 2 = a 1 +a l + 1 +a 2 i + 1 + .... 

B l = bi- i +b 2 i-i + b 3l - 1 + ... Ai = a,- l +a l - 1 +a 2 i- 1 + .... 

( 6 ) 

If the non-linearity is, for example, the saturation shown in 
Figure 4, then 

a ; =lfor|Xi|<zl, a;=—forx ; >d, a-= — 2forx ; <-A (7) 

As shown in this example, when the non-linearity is a single- 
valued function of x h then the weighting coefficients are uniquely 
defined as the functions of x t . Therefore, eqns (5) have / unknown 
quantities, x { or (i = 1 ~ /), and the number of unknowns 
coincides with the number of equations. In other words, 
eqns (5) give the equilibrium conditions for the non-linear 
oscillation of period / T in the system with the non-linearity 
defined by a single-valued function of the input pulses. 

Now if each value of input pulses x { is assumed, for example, 
as x x £ A, | x, | < A, x 3 < - A ,..., then substituting eqn (7) 
into eqn (5), yields a set of the first order simultaneous equa¬ 
tions of x { . When the solutions of the simultaneous equations 
satisfy eqn (7), the equilibrium state is established. On the 
other hand, if they do not satisfy, the equilibrium state as 
assumed above does not exist. These procedures must be 
applied for all sets of possible combinations of x t . For example, 
when / = 2, 3 and 4, the number of such sets is 2, 4 and 10, 
respectively. Thus, in general, the number of sets increases as / 
becomes larger, but the difficulty of the analysis is fortunately 
independent of the value of m, the order of the system. However, 
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any complementary condition such that, for example, the system 
has a pole at the origin of a complex plane, will be very effective 
in decreasing the number of sets. 

Stability of the Equilibrium State 

In general, the sampled output of a non-linear element can 
be expressed in the form: 

y*(t)==y*[x*(t)2 (8) 

Introducing this notation into eqn (3) gives 

z lx* CO] = -G* (z) Z {/* [x* 0 )]} (9) 

Assuming that the periodic equilibrium state is affected 
by small non-periodic disturbances, the input of the non-linear 

element * * ( t ) is superposed by 


Therefore, 
y*[x*(t)] = 

+ (t-T)-f £ 2 <5x 2 <5 (t-2T) + ... + ( t-lT) 

+ %i$ x i+(tT) + % 2 dx l+2 S(t — 1+2T) +... (17) 

Thus the stability problem of the equilibrium state can be con¬ 
verted to that of the sampled-data system with the periodically 
varying gain illustrated by Figure 6. 



bx* ( t ) = <5x 0 <5 (t) + 8x t S(t-T) + Sx 2 S (f —2 T)+... (10) 


Figure 6. Variable gain sampled-data system 


Then the sampled output of the non-linear element can be ex¬ 
pressed by y* [£*(*) + and eqn (9) becomes 

Z lx* ( t ) + <5x* (f)] = —G* (z)Z {y* [x* (t) + 5x* (f)]} (11) 




,x*(tK8x*(t) 


T 






y*rx*(t) + Sx*(t2 


Figure 5. System used for stability criterion 
Such a system is shown in Figure 5. Since clearly 

Z [** CO + Sx* (*)] = z lx* (0] + Z [<5x* (0] (12) 

substituting eqns (9) and (12) into (11) yields 

Z [<5x* (0] = — G* (z) Z{y*[x* (t) + Sx * (*)] - [x* (t)]} 


or 


(13) 


Z[8x*(t)J 


(14) 


The extreme value of this equation as the disturbance becomes 
infinitesimal gives 


Z [<5x*(*)] = — G* (z)Z {y* [x* (f)] 5x* (0} (15) 

where y* [£*(/)] is the derivative of y* [£*(/)] with the input 

pulse series x*(t), where x*(0 is a periodic signal with period IT 
and can be represented by the repetition of / pulses x x ~ 
When the non-linearity is, for example, the saturation shown in 
Figure 4 , y is represented as 

y*[x*(0] U=6, &=0 for |x f |>zd 

, i, , . „ (16) 

6=1 for IxJ < A 


Since the system shown in Figure 5 is a kind of linear system, 
the stability can be determined by the convergency of the re¬ 
sponse to a unit impulse disturbance. Denoting the value of 
the impulse response of the linear system at t - nT by g n , 


Now, if 


2 ujj 


G*(z) z"~ 1 dz 


(18) 


N (z) 

G *( z )=TTkr v M (z)=(z—r 1 ) (z —r 2 )... (z — r m ) (19) 


then 


go = 


M (z) 5 


GHz) 




2nj, 


dz — 


G(0)+ £ 


N(r t ) 


and 


1=1 r t M'(r,) J 


( 20 ) 


gn = 


2nj 


n = 1,2,... (21) 

*(r.)-*W 
,J M'(r t ) 

Denoting the input pulse series of the controlled element 

oo 

G*(z) by cf fi {t) ~Zi Ci d(t— iT ), and the sampled output due 

/=o 

to only the single input pulse by O l *(t), then, 


where 


Z [O'* (0] = G* ( z ) c t z 1 (22) 

For simplicity, the case of / = 2 is considered where the 
gain constants g t in Figure 6 take the two values 6 and £ 2 
alternately with the period T. The sampled error signal s„ at 
t = nT is as follows; 


n—0: 

e ° = ra SJi ’ Co= ^ £ ° (23) 

■'■ o0 *(t) = lg 0 S(.t) + X$d(t-T)+I$-r i 8(t-2T) 

+ X4>-r?8(t-3T)+...-]l; 1 A 1 
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n — 1: 

1 d + ^go) ?1 1 2 
where the symbol [ ] denotes a matrix, and 


■W[Mlp2 

(24) 


1 ^]! = I^K r l)> ^ ( r 2>> • • • > ^ ( r m )I (25 ^ 

U 1 A 1 J = U,U...,ll^A 1 [] t : transposed matrix 

.-.O 1 * (t) = [g 0 5(t-T)+md(t-2T)+mr i 8(t-3T)+...'] 

x[-'£4>Z 1 A 1 £ 2 A 2 '] 


s 2 = [ - MrfoA i +1 IQ^A A 2 ] A t 

=W [[<^2^2] - Ml Ki^i] ^ 1 

where 


ih 2 a 2 j= 

^Oh), $(r 2 ), .. 

■, f(rj 


_<$0t), ^( r 2>= ■■ 




0,. ... 

0 ’ 

0,. 

r 2 > ' • 

'0 

_0, 

O 



(26) 


(27) 


,\O 2 *(0 = 

[g 0 5(r-2T) + 2^(t-3T) + E^r ; 5(r-4T)+...J 
x [-E^ 1 ^ 1 +E^ 1 d 1 I^ 2 d 2 ]^i^i 


s 3 = [-2^/f+i:^r i E^ 2 2J 2 +S^r i S<P^ 1 dj 
-(L4>) 3 Mi£2^2]£dM2 

=M [-B/F+M W [^2^2] 

= - W [[ 0 ^ 1 ]-M] [[^ 2 ]- Ml Ki^tl 4d 

(28)* 

where 


l$ZlA l } = 

>0t), 4>(r 2 ), . 

■; $(r,n) 


&(r 2 ), . 

$(rj_ 


In general, 


(29) 


«2„+i = - M {[[^i^ri-W] [[^^2^2] - WlfKi^il ^2 


that is, the roots R ( of the equation 

det. W]} = ° < 31 ) 

where [17] denotes a unit matrix, the stability of the system m 
Figure 6 is determined as follows 1 . If all absolute values of the 
characteristic roots are smaller than unity, the system is stable 
and, if not, it is unstable. It is also possible to apply the Hurwitz 
stability criterion to this problem by introducing the following 
bi-linear transformation into eqn (31). 

(32) 

(A-l) 

In general, the stability criterion of the equilibrium state of 
period IT is given by the absolute values of the roots of 

det. {R [17] — [7 [WA i!-W]}=° < 33 ) 

i= 1 

Examples 2 

Example i—The system shown in Figure 7(a) is a finite-settling- 
time system for both a step input r(t) and a step disturbance d(t). 
A numerical example of the responses, when a single impulse of 
the magnitude y 0 is applied to the system, is shown in Figure 7 (b). 
From the diagram, it is seen that the oscillation ot period 
IT = IT = 0-5 is caused thereby and that it builds up when 
y 0 > 0*483 Id, but vanishes when y 0 < 0*4831/1. It is presumed 
that the unstable equilibrium state of the oscillation of period 
IT = 2Tmay be obtained when r 0 = 0*4813/1. 




1 0-25 


Zero order hold 



Controller D*(z) 


Saturation Controlled 
element system 


(a) 



Figure 7. (a) A finite-settling-time response system (1); (b) Transient 
response to an impulse input y ( t) ~ y 0 8 (t) 


«2» =M[[^C2^2l-W] 

X {mi^J-MM^^J-lrW-^AjA, (30)** 
Therefore, by using the characteristic roots of the matrix 


* \4> h Ai\ frj ^ [rj 14 >Is A t j, but [ <j>}4> l« M H ^'1 

= l<p\ W 10 Si di] I Si di]. . . 

** In the equations, ^ 0 is assumed. If Si — 0, the system is 
never disturbed, and then S 2 must be substituted for If Si = 0 and 
f 2 = 0 at the same time, then all input pulses x l9 x 2 are saturated and 
the stability of the oscillation depends on whether G* (z) D F * 00, the 
open-loop transfer function in the original system, is stable or not. 
t DM> l 2 A& - IrJ] Dt4» h xlj - Ml may be also used. 


Such phenomena are analogous to ‘the hard self-excitation 
in continuous systems 3 . 

In this system, 

B x = 1-3853, A x = -1-3853, B 2 = - i-3853, A 2 = 2-3853 (34) 

and the following two cases are considered as the possible 
equilibrium states: 

(1) x 1 >A,x 2 <-A(:.a 1 x 1 = A,oc 2 x 2 =-A) (35) 

In this case, eqn (5) becomes 

B l x 1 + B 2 x 2 = -(A 1 -A z )A, 
B 2 x 1 +B l x 2 =(A 1 .<4 2 ),d (36) 
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.'.Xi — x 2 = 2-72 A (37) 

Here, two equations (36) being linearly dependent, each value 
of x x and x 2 cannot be uniquely determined by eqn (36). As 
illustrated in Figure 8 , the solutions which satisfy eqns (35) and 
(36) exist infinitely in the intervals of 

A <x t < 1-72— i*72zl <x 2 < —A (38) 



Figure 8. Periodic equilibrium state 


This suggests that the mean value of the controller output 
should be given as another complementary condition, because 
the controller D F {z) has a pole at z — 1 and the magnitudes of 
both pulses x 1 and x 2 are limited to constant values + A by the 
saturation element independently of the mean value of the con¬ 
troller output, and each value of x ± and x 2 can be uniquely de¬ 
termined when the mean value of the controller output is given. 

(-2) ^ A , |^21 ^ A (. .oqx^ = zl, oc 2 :=: 1) (39) 

B 1 x 1 + (J3 2 + A 2 )x 2 — —A t A 
B 2 x 1 +(B 1 + A 1 )x 2 — -A 2 A 

:.x 1 = 1-72 A, (cc 1 N 0*585), x 2 = - A , (a 2 = 1) (40) 

The above solutions are limiting values of eqn (38). 

Consider the stability of these equilibrium states. In this 
system 

= Ut 2 -= 0-779, r 3 = -1 -779, A x = A 2 = 1 
0 Or) = 1*628, <f> (r 2 ) = - 0-836, 0 (r 3 ) = - 0-792 ' j 

and 

Ci =0and £ 2 = 0forx 2 ^A,or0<£ 2 <lfoix 2 = - A (42) 
.‘.det. {* [C7] - im.A,} - [r]] [l<f>Z 2 A 2 ] - [r]]} 

= R 3 - (4-772-1 -395 c 2 ) R 2 + (5-693 -3-328^-0-313 ft) R 
+ (-1-921 +1-123 £ 2 +0-313 £ 2 —0-299 £f) = 0 

-(0-810 + 0-299 f 2 )^ 3 (43)* 

+ (1-702 +1-354 £2 - 0-626 + 0-897 £|) A 2 

+ (-3-684+5-302 £ 2 +1-252 ■-0-897 {f ) A 

+ (13-39 - 5-846 c 2 - 0-626 + 0-299 ) = 0 

By considering the signs of coefficients of A 3 , A 2 and A it can be 
concluded that eqn (43) has at least one unstable root for 
0 < £2 < 1 and, hence, that this equilibrium state is unstable. 


* When the saturation characteristics are idealized as shown in 
Figure 4 £$ changes at | x% | = A discontinuously, but changes 
continuously from 0 to 1 for the practical element. 


Example 2—Figure 9{a) shows another finite-settling-time 
response system with the delay time IT = 0-5. From the nu¬ 
merical examples illustrated in Figure 9 (b) the presence of the 
unstable equilibrium state of period 3 T = 0*75 may be estimated. 
By the same procedure as in Example 7, the following unstable 
periodic equilibrium state of period 3Tcan be confirmed, 

x 1 = l‘55A>A, x 2 =—A, x 3 =0 (44) 



(b) 

Figure 9. (a) A finite-settling-time response system (2); (b) transient 
response to an impulse input y (t) = y 0 S it) 


Describing Function for Non-linear Sampled-data Systems 

For the analysis of steady-state phenomena in non-linear 
sampled-data systems, a so-called describing function can be 
introduced as a useful tool as in continuous systems. However, 
the most important subject, in this case, is the method by which 
the sampler must be treated. 

The First Method 

A sustained oscillation of frequency co in the non-linear 
sampled-data control system is assumed as illustrated in Figure 10, 


/ y*(t) 



Figure 10. Non-linear sampled-data system 


Now assuming that o> co s , where m s = 2njT is the sampling 
frequency, and that the sampled controller output c*(t) is 

r-s-' CO 

c*(t)=c Y sin (wiT+<p e ) <5 (t-iT) (45) 

i— — oo 

then the continuous output of the non-linear element y(t) and 
its sampled pulse series y*(f) arc approximated by 

y(t) = N(x)x sin (cot + cp c + <p h ), 

x = \H{jco)\a, s C\2 n, cp h =<H (jw) (46) 

y*(t) = N(c)c sin (wit+(p c ) 5 (t-iT) (47) 
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where N(x) and N(c) are the describing functions of the satura¬ 
tion element and j is)'—1. N(c) is a describing function for the 
sinusoidal envelope of controller output pulse series, and N(x) 
is that for the fundamental sinusoid of staircase wave of the 
hold-circuit output as illustrated in Figure 11. Corresponding 
to eqns (46) and (47), the two kinds of balancing equations for 
self-sustained oscillations are given, respectively, as follows; 

l + jV(x)<K(®) = 0, 

<M©) = (e J 2K fs) H (jeo) G (jo) (48) 

l + N(c)(j>X(o)=0, 

CO CO 

(j>X ( 0 ))= D't (e J 2n HG* (e J 2,1 «,) (49) 



Figure 11. Hold circuit output waveform 


The phase angle <p c is not contained in cqns (48) and (49); in 
other words, the equilibrium states are determined independently 
of the phase angle. Consequently, the accuracy of this method 
decreases as co tends to co s and the sampled signals such as 

c*(t) and y*(0 cannot be determined unless the other additional 
conditions concerning the phase angle are given. 

The comparison of the utility of the above equations is given 
in the following examples. 

Example 3 —In Figure 10, supposing that the non-linear 
element N is the saturation element as shown in Figure 4, and 
that the transfer functions Gis), H(s) and D F (z) are given by 

G ^=^W r =°- 25 

68-509 — 74- 104z~ 1 +23-678z~ 2 } 

AZ 1—0-19447 z -1 —0-86553 z -2 

then the system is a finite-settling-time system. Figure 12 
shows a numerical example of the controller output, when a step 
function of magnitude y is applied as the input y(t). The ‘hard 
self-excitation’ takes place also in this system, as illustrated in 
Figure 12. Figure 13 shows Nyquist diagrams by eqns (48) and 
(49). In the figure, the cross point of the frequency vector locus 
^(co) and the amplitude locus — l/N(x) coincides with that of 
(j)l (co) and - 1 IN(c). The cross point is an unstable equilibrium 
point. The value of the point is coT = 0*57, (that is oj ^ cojll) 
and C = x = — 4*45. The broken lines in Figure 12 indicate the 
equilibrium amplitude C = x = 4*45. These values of co and C 
(or x) can fairly explain the phenomena shown in Figure 12. 


A 



Figure 12. Transient response 



Figure 13. Frequency and amplitude vector loci 


Example 4—Figures 14 and 15 show Nyquist diagrams with 
the amplitude locus of -l/N for the systems illustrated in 
Examples 1 and 2 respectively. In Figure 14, the vector locus of 
^(co) does not cross the amplitude locus and so the presence of 
any equilibrium oscillation cannot be expected. But both vector 
loci of ^(Wi) in Figures 14 and 15 have the unstable cross point 
coinciding with the amplitude locus; hence, in this case, the 
presence of the unstable equilibrium state can be expected. 
However, the numerical values of co and C given by the cross 
points differ from the accurate values given earlier. 

The Second Method 4 

A describing function for co ^ co s is introduced by con¬ 
sidering the phase angle of the oscillation at the sampling in¬ 
stants. To simplify the analysis, the following restrictions should 
be applied. 

(1) G(s) or D f (z) is assumed to include at least an integrative 
pole. From this assumption, it can be concluded that the mean 



Figure 14. Frequency and amplitude vector loci 
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value of y(t) over a period must be zero tor tfte continuation of 
the periodic phenomenon, and that the ratio cojco is an integer 
number such as 2, 3, 4 etc., because the existence of a zero-order 

hold makes the sign changes of y(t) take place only at sampling 
instants as illustrated in Figure 16. 




(2) The periodic phenomenon is assumed to be a sinusoidal 

oscillation of frequency co. When c*(t) passes through the zero 
order hold and the saturation element, the output y(t ) becomes 
a staircase wave as represented by hatching in Figure 16. De¬ 
noting the fundamental component of the output y(t) by y(t ), 


(1) cojco = 2—In this case evidently 


0<xl<csin (p c 

• c . - 

• • ^mm " 

A 

~ sin cpj 

^max ~ 00 

(53) 

1 

7lCQ J(f>c 

1 



1 ’ „ r\, n _ _ ft 

N 2 (c, cp c )~ 

4A ’ 

n 2 

Omin 

4 sin cp c 

I0l0(p< c < — 

1 

nce J,fc 

1 


_ n _ 

N 2 (c, cp c )~ 

4 A ’ 

N 2 

O'mm 

— . . 1U 

4 sm (p c 

1 

/ r a\ 


Figure 17 is a vector representation of eqn (54). The frequency 
locus cpjco) plotted in the diagram is the same system illustrated 
in Example 1. From this, the point corresponding to a) s /co = 2 
falls inside the — 1 /N 2 domain and its coordinates give the 
periodic equilibrium state of period 2 T. 



Figure 17. Vector representation of — l/N 2 


(2) coj cd = 4—Mode 1: The saturating state in which a positive 
and a negative pulse in a period are saturated as indicated in 
Figure 16. In this case 

Csin cp c <A<C$m( cp c -\-^-\ C m5n =—-— C =— (55) 

\ 2/ mm cos<p c ’° max sin<?/ 

The function — l/iV 4 for Mode 1 can be similarly defined and its 
vector representation becomes as indicated in Figure 18 {a). 
Mode 2: The saturating state in which all the pulses in a period 
are saturated. 

In this case 

C<A<Csm(p c :.C min =^~-, C max = +co (56) 


7(t) = Im (N asia (c, <p e ) c t J (o,t+ ^) (51) 

where No Jo (c, (p c ) is a describing function of the saturation 
element combined with a zero order hold. It is a function not 
only of c and cp c but also the value of cojco and the number of 
saturated pulses in a period. The balancing relation in this case is 

1 + Ncoja) (c, cp c )cj) 2 (co) = 0, cf) 2 (co )=Dp (e j 271 G (jo) (52) 


The vector representation is indicated by Figure 18(b). 

The — 1/M 4 domain of Mode 2 coincides with that of Mode 1. 

The Case cojco is an Odd Number —When cojco is an odd 
number, the interval of reasonable values of (p c is not 
— co772 < (p c < co T/2, as in the case of an even number of cojco, 
and takes peculiar values corresponding to cojco. For example, 
if cojco = 3, cp c = 0 is the only possible value which satisfies 
the two restrictions. In this case 


The Case cojco is an Even Number —When cojco is an even 
number, the output of the saturation element has no d.c. com¬ 
ponent independently of the value of cp c and practically, by con¬ 
sidering only the region of - coT/2 < cp G < coT/ 2, the phenomena 
can be estimated. 


0<zl<Csin 


2 tt __\/ 3 r 
3 2 


:.c 


min 


^3 ^ 5 ^max +00 


(57) 


1 _ 7 iC jn 

_ ii e 3 ’ 


1 

N 3 


On in 


271 fit 

“T/3 e5 


(58) 
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Figure 18. Vector representation of — 1/JV 4 : 

{a) Mode 1; (b) Mode 2 

Evidently the vector locus of eqn (58) is a straight line as indi¬ 
cated in Figure 19, and the frequency vector locus c/> 2 (co) in the 
diagram is the same as that in Example 2. The point corre¬ 
sponding to cojo) = 3 is very near to the locus of — I/A 3 , but 
not on it. This discrepancy is due to restriction (1). That is, the 
actual waveform is quite different from the sinusoidal wave as 
indicated in Figure 9(b). 

Conclusion 

The analysis described first does not include any assump¬ 
tion and is applicable to non-sinusoidal oscillations and to the 



phenomena in higher order systems. But this analysis is restricted 
to the phenomena of period IT where / is an integer; and when / 
is increased, the analysis becomes extremely difficult. And more¬ 
over, to estimate the presence of the oscillation, the analysis 
must be carried out for various values of /. 

By the first describing function methods proposed, the 
periodic equilibrium states and their stabilities are very easily 
analysed. But the accuracy of the results decreases rapidly as the 
frequency of oscillation approaches the sampling frequency, 
because, in these methods, the waveform is assumed to be 
sinusoidal and the phase angle of oscillation is not taken into 
account. 

In the second method, the phase angle of the oscillation at 
the sampling instants is taken into consideration, and this 
method is applicable to the case where both frequencies ap¬ 
proach closely. But in this method, the assumption that the 
system includes at least one integrative pole is required in ad¬ 
dition to the usual assumptions in a describing function method. 
Thus the stability of the equilibrium state cannot be dealt with 
by this method. 
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DISCUSSION 


J. Pikielny, Przemystowy Instytut Telekomunikocji , Warszawa, 
Biatowieska 19/47, Poland 

Describing function methods have been used for analysis of linear and 
non-linear sampled-data systems for some time. This method is also 


mentioned in your paper, and I therefore ask if you have made any 
attempts, either theoretical or empirical, to estimate the accuracy 
of this method and what results were obtained. It is very interesting 
to compare the results obtained by first and second (Chow’s) describ¬ 
ing function methods. 
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B. Kondo, in reply 

Thank you for the discussion of our paper. The first method is clearly 
less accurate than the second one. The main reason might be the lack 
of the phase difference between the sustained oscillation and the 
samples in describing function N (c) or JV (x). 

For example, consider the extreme case where the saturation level 
is very small compared with the amplitude of oscillation. In this case, 
it can be seen that the output is always the same when any sinusoidal 
input with an arbitrary phase angle between — njl and njl is applied. 
This means that the phase angle of describing function varies with 
the phase difference between the samples and the oscillation (this is 
very clear in Chow’s method). Thus, roughly speaking, it can be seen 
that the maximum phase error is ± njl for the saturation element. 
Similarly, the gain error of describing function can be considered. 

However, the above conclusion is unfortunately limited to the 
curve where o>/ca s is an integer, because the describing function in the 
strict meaning cannot be defined in our system 1 . This problem remains 
to be solved in the future. 

Reference 

1 Kalman, R. E. Non-linear mechanics Symp., Polytechnic Inst. 
Brooklyn U.S.A. (1956) 

E. Pavlik, Siemens and Halske , Karlsruhe, Lassallestr. 9, W. Germany 

The analysis is restricted to the phenomena of period IT where / is 
an integer. 

In Example 1 it was empirically stated that there was such an 
integer IT. Must this integer always be stated empirically or does 
an analytical method exist? 

B. Kondo, in reply 

We are sorry to say that, as far as we know, the method of determining 
directly the possible value of / has not yet been found. In our method, 
assuming the value of /, eqn (5) can be solved. If a reasonable solution 
can be obtained, then it can be concluded that the oscillation with the 
assumed period exists. 

We would add, however, that Professor Jury and Dr. Nishimura 
have succeeded in formulating the equation by which the possible 
maximum period of the oscillation can be determined under some 
restriction for PWM systems. 

M. Hamza, ETH , Zurich , Switzerland 

Could Professor Kondo please indicate whether his method can be 
modified to cover systems with hysteresis ? If not, which method does 
he suggest ? 

B. Kondo, in reply 

Equation (5) in our paper can also be applied to the system with 
hysteresis. In this case, we need the assumption for formulation of 
eqn (5) that the operating point corresponding to a given xt is on either 
the upper region or the lower region of the hysteresis. Under this 
assumption, we can formulate the equation. However, it is necessary 
to examine if the obtained solution of eqn (5) satisfies the above- 
mentioned assumption. 

A. Lepschy and A. Ruberti, I.S.P.T. Viale Trastevere 189, Roma, 
Italy 

We intervene in the use of the describing function for the analysis of 
limit cycles, neglecting the first method which, as the authors them¬ 
selves affirm and which Tsypkin 1 has shown, holds only if the rate 
(o s lo) is high. As far as the second method is concerned, we feel that 
it will be useful to communicate our studies with an analogous 


procedure in a more general case 2 . Taking into account the fact that 
the method applies only if the block which contains the sampler and 
the non-linear element is represented by one describing function 
only 3 ” 6 , we proposed to elaborate a simple procedure for the analysis 
of a generic instantaneous piecewise non-linearity. 

We have therefore considered the sum of a number of dead zones 
or of limiting characteristics instead of the actual non-linear character¬ 
istic; then we have studied the approximation limits within which 
it is allowed to sum the describing functions of the elements (formed 
by a sampler, a holder and a dead zone or a saturation) the parallel 
of which is substituted for the block formed by the sampler, the holder 
and the actual non-linear element. 

We have preliminarily investigated the case in which the non¬ 
linear element is a dead zone or a saturation. (The latter case is the 
same as that studied by Messrs. Kondo and Iwai.) In this analysis we 
thought it necessary to take into account the effects of the continuous 
component at the input of the non-linear element (the continuous 
component at the input is there if the rate co s lco is even and G(s) has 
a pole at the origin, which prevents the non-linear element from having 
a continuous component at its output). The study of the amount of 
the continuous component 7 and of its effects, is important with regard 
to the above-mentioned decomposition of the actual non-linear 
element. In our paper we present the normalized diagrams of the real 
and imaginary parts of the describing function of the block formed 
by sampler, holder and dead zone. Each diagram is calculated for one 
value of cos/o) (from 2 to 10) and shows how the describing functions 
depend on the amplitude of the input signal and on its phase with 
respect to the sampling impulses. The diagrams give the real and the 
imaginary parts of the describing functions in order to facilitate their 
composition. 
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B. Kondo, in reply 

Thank you for your very valuable comments on our paper. The 
results achieved with your investigations of similar problems are very 
useful, and it is hoped that they will be of assistance in our study 
in the future. 


G. Senouillet, R. Boudarel and P. Guichet, Centre d'Etude et de 
Recherche en Automatisme, 32, Boulevard Victor, Paris 15° France 

The research of the conditions of existence of periodic limit cycles 
is greatly simplified by the use of a sequence vector which provides 
a unifying concept to the methods proposed by different authors. 
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A sampled periodic oscillation is represented by a vector in an 
orthonormal space the coordinates of which are the values of the 
successive samples. The number of coordinates of the vector is equal 
to the number of samples included during one period of oscillation. 

Then, considering a non-linear sampled data system consisting 
of a number of linear elements separated by instantaneous non- 
linearities, and assuming that there exists a limit cycle of period NT, 
the corresponding oscillation can be represented by a sequence 
vector E defined numerically or parametrically at a given point of the 
loop. Opening the loop at this point it is possible to compute the 
transmission of the sequence vector E throughout the loop. The 
assumed oscillation will exist when the output sequence vector S 
will be identical to the input sequence vector E . This yields the form¬ 
ulation of sufficient and necessary conditions of existence of the 
assumed oscillation. 

The practical problem arises when the computation of the trans¬ 
mission of the sequence vector is to be performed. For this, the 
sequence vector can be given several representations, depending upon 
the choice of the bases of the orthonormal space. 

One type of representation is transient, another type is harmonic. 

The time representation may be direct, i.e. the successive coordi¬ 
nates are the successive samples of the oscillation; or inverse, i.e. the 
successive coordinates being taken backward of the successive samples, 
coordinate zero being the value of the signal at the sampling instant 
chosen as origin, the coordinate number one being the value of the 
signal at the former sampling time, etc. The harmonic representations 
are defined in relation to the direct time representation. 

One can consider first a base, the vectors of which are 

l4 = cos ^, m integer > 0 

where is the kth coordinate of vector with respect to the direct 
transient base. Then a periodic oscillation of period TV can be repre¬ 
sented by the limited expansion 

2 7 i/c . Ink 

s(k) = a 0 +a t cos — — Va 2 sm—... 

where the ai's are the coordinates of the sequence vector with respect 
to the new base. 

One can also consider the direct transient space as Hermitianand 
choose a new base the vectors of which are 



which yields a closely-related representation of the sequence vector. 

Depending upon the chosen representation, several methods can 
be developed for computing the transmission of the sequence vector 
throughout a loop consisting of linear sampled elements separated 
by instantaneous non-linearities. 

First it is to be noted that only the time representation yields a 
straightforward expression of the action of the non-linearities since 
it appears successively on each sample, that is, on each coordinate. 

If the non-linearity is given by 

s-f(e) 

where e is the input of the non-linearity and s is the output of the 
non-linearity, the kth coordinate of the output sequence vector 5 
will be related to the kth coordinate of the input sequence vector t by: 

s k = f(e k .) 

In the other representations, where coordinates are not successive 
samples, the effect of the non-linearity cannot be separated for each 
coordinate. It is therefore convenient to come back to the time repre¬ 
sentation by performing the coordinate transformation. 


The transmission of the sequence vector through a linear sampled 
element can be computed in time representations and in harmonic 
representations. 

Transmission of the Sequence Vector in Time Representation 

The output of the linear element is constituted by an oscillation of 
the same period as the input. Therefore, the transformation in the 
sequence vector space yielding the output vector S, starting from the 
input vector E, is a linear transformation. 

S=WE 

where IF is a matrix. 

The choice of the inverse time representation yields a particularly 
simple structure of th transmission matrix which has only real 
elements. 

Naturally the presence in the linear element of a pole at 1 makes 
that the transformation degenerates and special treatment must be 
reserved to the integration factors which would be separated from the 
rest of the element. 

Such a matrix has interesting properties 1 , for example, (a) They 
are associative; (b) They are commutable; (c) They are invariant in 
time translation for stationary filters and therefore are circulant, 
i.e. invariant in circular permutations of lines and columns; and 
(d) Noting P as the circular permutation matrix, the sampled transfer 
function Z" 1 corresponds to the transmission matrix P and therefore 
the filter F{Z~ x ) “ £/*■ Z~ ? ’ has for transmission matrix 

W^ftP^FiP) 

Matrices W can be computed by using Z transform techniques and 
the use of the skip sampling theorem 2 greatly simplifies the evaluation 
of the integrals involved. The Frazer-Duncan-Collar formula can be 
used in order to compute directly 

W=F(P) 

Matrix techniques provide a straightforward method to compute 
the transmission matrix IF 1-3 . The transfer function being written 
under the form of parallel or cascaded expansion, this method yields 
the transmission matrix W for the elementary elements 


The general transmission matrix is the combination of the elemen¬ 
tary transmission matrices. 

Transmission of the Sequence Vector in Harmonic Representation 

The representation in an Hermitian space yields a diagonal maliix 
the terms of which are complex numbers 4 - 5 , while the representation 
in an Euclidian space yields a real anti-symmetric matrix 3 . 

From this last representation, if the linear element is a low-pass 
element, it is sufficient to limit the expansion of the input and output 
sequence vectors to the two first coordinates. This constitutes the first 
harmonic approximation which was the first developed for the analysis 
of non-linear sampled-data systems 7 - 8 . Considerations upon the 
representations of the sequence vectors show that, when the first 
harmonic hypothesis is valid, the first harmonic approximation is 
identical to the sequence vector itself. Therefore, there is no necessity 
to compute any harmonic decomposition of any sequence vector, 
the time representation being perfectly adequate*. 

In sampled-data systems, because of the folding of frequency 
caused by sampling, the domain of application of first harmonic 
method is more restricted than in continuous systems. Moreover, the 
first harmonic approximation, when valid, provides only a method 
for sinusoidal type oscillation and it is well known that non-linear 
sampled-data systems exhibit non-sinusoidal type oscillations. 
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In fact, rigorous methods do not necessitate computations more 
intricate than the first harmonic approximation. Moreover, the matrix 
techniques can easily be mechanized and therefore allow one to cope 
with high-order systems with any number of non-linearities, as is shown 
in Reference 1 in the particular case of coded non-linearities. 

The preceding considerations will be developed at some length in 
a paper ‘Analytical Study of Non-Linear Sampled Data Systems’, by 
R. Boudarel, P. Guichet and G. Senouillet, to appear in Progress in 
Automatic Control , Vol. 3 (edited by MacMillan). 
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R. E. Kalman, Research Institute for Advanced Studies , 7212 Bellona 
Avenue , Baltimore 12 , MD ., U.S.A 

Since this is a carefully written paper, it is especially unfortunate that 
the authors seem unaware of the extensive literature concerning this 
problem. 

It is well known 1 that Chow’s method can be expected to hold only 
under highly restrictive assumptions. Both Chow’s method and the 
first method of the paper were thoroughly explored by Torng 2 . I am 
obstinately of the opinion that transform methods are all wrong for 
non-linear problems. The ‘natural’ mathematical approach is always 
in terms of state variables, the success of this approach is amply clear 
from the literature. 

References 

1 Kalman, R. E. Non-linear Mechanics , Symp. Polytechnic Inst. 

Brooklyn. U.S.A. 1956 

2 Torng, J. and Meserve, M. E. Trans. Amer. Soc. mech. Engrs ; J. 
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B. Kondo, in reply 

We would not say that our paper was the first in which the describing 
function was applied for the analysis of non-linear sampled-data 
systems. The main purpose of the paper lies in the proposition of the 
strict method, the description of which covers the first two-thirds 
of our paper, and I would be grateful if readers will bear this in mind 
when reading our paper. 

Ya. Z. Tsypkin, Institute of Automation and Telemechanics , Moscow 
1-53, Kalachevskaya, 15-A , U.S.S.R. 

The interesting paper by Messrs. Kondo and Iwai gives precise and 
approximate methods of determining periodic behaviour in non¬ 
linear sampled-data systems. A number of studies have been made in 
this field in the Soviet Union, and have been published in the journal 
Automat. Telemech. from 1960 to 1963, and in the proceedings of a 
symposium on the theory of non-linear oscillations, Kiev, 1961. 
In a number of cases the results obtained differed from those of 
Kondo and Iwai only in form. 

I should like to note that for determining symmetrical non-periodic 
behaviour in non-linear sampled-data systems it is more convenient to 
use, in place of the simultaneous eqns (2) and (3), the system 

N-l 

n=0, 1,2, ...,N ( 1 ) 

v = 0 

where 

(a 

where W* (jco) is the frequency characteristic of the linear sampled- 
data side and N the half-wave of the oscillations. 

This system can easily be solved by iteration, and x [u] thus 
determined. 

The stability of forced periodic behaviour im Grosser: can be 
studied on the basis of the approach described in our paper read at 
this Congress. 

I should like to mention the very interesting and detailed examples 
in the paper by Messrs. Kondo and Iwai, which reveal a number of 
phenomena in non-linear sampled-data systems. 

B. Kondo, in reply 

Thank you for your valuable discussion of our paper. By introducing 
the relation 

yi=.X t = ( Xi ) 

where (x) denotes the non-linear characteristic, eqn (5) in our paper is 
rewritten in the form of 

Bx= —Af (x) 

The above form is similar to one which you have shown, but the 
meaning of coefficient matrices A and B is quite different from yours. 


164 



Continuous Compensation of Feedback Sampled-data 

Linear Control Systems 

B. M. BROWN 


Summary 

One of the more important applications of the theory of sampled data 
control systems is that of the design of a compensator for a feedback 
system when the characteristics of the plant are specified. This prob¬ 
lem is relatively simple when a discrete compensator is used. It is, 
however, much more difficult in the case of a continuous compensator 
because of the impossibility of separating in the overall operator or 
transfer function the contributions of the plant and compensator oper¬ 
ators. Such solutions as have been proposed either use approximate 
methods, or introduce extra samplers or subsidiary feedback loops 
into the system. In this paper a graphical method is described for 
determining the operator of a simple continuous compensator for a 
given plant operator, assuming that the order of the system and the 
poles of the overall operator are specified. It is assumed, moreover, 
that the compensator must be physically realizable with a plant whose 
load has finite inertia. 

Sommaire 

Une des applications lcs plus importantes de la thSorie des systemes a 
donnees echantillon^es consiste it calculer un compcnsateur pour un 
systeme a contre reaction, les caracteristiques de Finstallation etant 
donnees. Ce probleme est relativement simple quand on fait usage 
d’un compensates discret, mais il est bcaucoup plus difficile avec un 
compensates continu, parce qu’il est impossible de s£parer les con¬ 
tributions de Foperateur de Finstallation et de celui du compensateur 
dans Foperateur global. Les solutions existantes se basent sur les 
methodes approximatives, ou b:en dies introduisent des echan- 
tillonneurs supplemcntaires ou boucles de reaction subsidiaires dans 
le systeme. Cet ouvrage decrit unc methode graphique pour determiner 
Foperateur d’un compensateur simple continu pour un operates 
d’installation donne en assumant que Fordre du systeme et les poles 
de foperateur global sont specifies. On assume en outre, que le com¬ 
pensateur doit etre physiquement realisable avee une installation 
dont la charge possedc une inertic finie. 

Zusammenfassung 

Eine der Hauptanwendungen der Theoric der Abtastregelsysteme ist 
die Berechnung eines Kompensationsgliedes ftir ein Regelsystem, wenn 
Eigenschaften der Anlage vorgegeben sind. Mit einem diskreten Kom- 
pensationsglied ist das Problem verhaltnismaGig einfach; die Verwen- 
dung eines kontinuierlichen Kompensationsgliedes ist jedoch viel 
schwieriger, da sich die Gesamtiibertragungsfunktion nicht in diejeni- 
gen der Regelstrecke und des Kompensationsgliedes aufspalten laBt. 
Die bisherigen Losungen beruhen entweder auf Naherungsverfahren 
oder auf der zusatzlichen Einfiihrung von Tastern oder Ruckfiihrungen. 
Die Arbeit beschrcibt eine graphische Methode, mit der sich die t)ber- 
tragungsfunktion eines cinfachen kontinuierlichen Kompensations¬ 
gliedes bei einer vorgegebenen Ubertragungsfunktion der Regelstrecke 
ftir den Fall bestimmen laBt, daB die Ordnung des Systems und die 
Pole der Gesamtubcrtragungsfunktion bekannt sind. Ferner wird 
angenommen, daB das Kompensationsglied fur eine Regelstrecke, 
deren StorgroBen Verzogerungen (Tragheit) besitzen, physikalisch 
realisierbar ist. 


Introduction 

The problems of the design of compensators for linear feedback 
sampled-data control systems have received much attention 
during the last 10 years 1 " 5 . In general it has been found that 
when discrete compensation is specified, relatively simple mathe¬ 
matical processes are available to determine suitable forms and 
parameter values for the compensating elements. The problem 
is much more complicated however when continuous compen¬ 
sation is used. This is due basically to the fact that the equivalent 
difference operator or pulse transfer function of two continuous 
linear elements in cascade without an intervening sampler bears 
no simple relation to the operators or transfer functions of the 
separate elements. For this reason it is not easy to determine a 
compensator which, operating on a given plant, will provide an 
output with specified characteristics. The methods that have 
been proposed for achieving this have either been approximate, 
or have involved trial and error, or have entailed the introduction 
of extra samplers or feedback loops into the system. 

This paper is perhaps not the place for a detailed discussion 
of the relative merits of discrete and continuous compensation. 
Certain of these will depend on practical rather than theoretical 
considerations and will vary with different applications. It can, 
however, be stated that with discrete compensation the signal 
directly controlling the plant can be changed only at the sam¬ 
pling instants, whereas with continuous compensation it is pos¬ 
sible for this signal to be varied continuously. In this way it is 
possible at the design stage to effect greater control over the 
response of the system between the sampling instants. In par¬ 
ticular, by suitable choice of the order of the system it is possible 
to reduce or eliminate the steady-state ripple in the output for 
certain specified inputs. 

One effective method for designing a discrete compensator is 
to specify the order and characteristic equation of the system 
and to deduce the poles and zeros of the compensator operator 3 . 
The subject of this paper is the analogous problem with a con¬ 
tinuous compensator. Without making any approximations it is 
shown how and under what conditions a physically realizable 
compensator of this type can be found. A special case of a 
problem of this type has, in fact, been discussed by Linvill and 
Sittler 8 . 

Operational Notation 

Operational notation and methods are used in this paper, 
the basic symbols being defined as follows. As usual, the oper¬ 
ator D applied to a function of continuous time t denotes dif¬ 
ferentiation with respect to t. When dealing with functions which 
are defined only for discrete equally spaced values of t, that is 
with sequences u T where r takes integral values only, two basic 
operators are used. These are the shift and backward difference 
operators E and V, given by the following relations: 


165 



B. M. BROWN 


Eu r = u r+l ; ^u r = u r — u r _ l 

so that V = l-£ _1 (1) 

In order to transform a function u(t) into a sequence u r , where 
u r = u(rT ), the sampling operator S is used. Thus 


Su ( t ) = u r = u ( rT ) (2) 

The constant T is the sampling period and in this paper is taken 
to have the value unity. Conversely, the first operation in the 
conversion of a sequence u r into a function of t is that of pulsing, 
denoted by P, where 


Pu r = £ u r 8 (t — rT) (3) 

r = - oo 

S(t) being the unit impulse function. By suitable combinations 
of these operators it is possible, in general, to represent un¬ 
ambiguously the relations between the various signals in any 
linear system which operates in part continuously and in part 
intermittently and which, except for the sampling that occurs, 
is time-invariant. 

It is convenient to introduce two other operators. The first, 
C denotes the process of clamping, that is, a combination of 
pulsing and a zero-order hold. Thus 

C = VD~ 1 P (4) 

The second is the attenuated difference operator defined by 
the relation 

V x =l-e~* T E~ x 

Then if T= 1, 

V^e^CV+Z) (5) 

where 

/=e A —1 (6) 

If a sequence u r is pulsed and the resulting pulse train is passed 
through an element with operator F(D), the output of which is 
sampled, the resulting sequence x r is given by 


x r =SF(D)Pu=F p (E)u r 

where 


F p (E)=SF (D)P (7) 

Alternatively, if the original sequence u r is clamped, then 
x r = F c ( E ) u r , where 

F c (E) = SF (D) C (8) 

HH_ x ' 


FAE) F(D ) 
V * D 


P 


In particular it can be shown that if F(D) = 1 /(£> +A), then 
F t (E) = 1/V A ; also that if FJE) = 1 /(V + /) a possible form of 
F(D) is given by 


F(D) = e~ A 


A 

l (Da-A) 



(9) 


A fuller account of the use of operators in linear system 
analysis has been given elsewhere 6 * 7 . It may be mentioned that 
if, in this paper, D is replaced by p or s and E by z, the theory is 
given analogously in terms of the Laplace and z transformations. 
In particular, F P (z) is the pulse transfer function of an element 
whose transfer function is F(p). 


General Theory 

The method which will be used for synthesizing the sampling 
feedback control system to be considered may be appreciated 
more clearly if it is compared step by step with that for an 
equivalent continuously operating system. Consider therefore 
the system of Figure i, with input u(t), output x(t\ error c(t) 
and open-loop operator A (D). The latter will be the product of 
the plant and compensator operators. It will be of the form 
P 1 (D)/D S Rj (D), where PjiD) and ^(D) are polynomials in D. 
Moreover, if the load on the plant has appreciable inertia the 
degree of the denominator of A(D) will exceed that of the 
numerator by at least two. The error is given by 

c(,) -i+7cDj“ w -^7$r" w (>°) 

where 

Q 1 (D)=D s R 1 (D) + P 1 (D) ( 11 ) 


U(t) c(t > 1 

A(D) 

1 x(t) 


r ' 




Figure 1 

The degree of Q X (D) will be the same as that of D'R^D). The 
stability and transient modes of the system depend on the zeros 
of the polynomial Q X (]D). The significance of the constant s 
(called the order of the system) lies in the fact, immediately 
a PP a rent from eqn (10), that the steady-state error is constant 
if D s u(t) is constant and is zero if any lower order derivative of 
u(t) is constant. 



Figure 2 


Consider now the system shown in Figure 2, in which the 
error is sampled and clamped before being applied to the com¬ 
pensator and plant. It can be shown that the error sequence is 
given in terms of the sampled input by the relation 

1 YR(V) 

° r 1 +A e (E) U '~ Q(V) " r (l2) 

where, as with the continuously controlled system, 

A c (E) = P (V)/W S R (V) 

and 

Q(V) = V S F(V) + P(V) 

Then for each factor D + A in R^D) there will be a factor V A in 
F(V). As above, the nature of the error will depend on the order 
s of the system and the steady state and transient response will 
be determined by the zeros of the characteristic polynomial Q(V). 

The problem to be considered first is as follows. Assume 
that these zeros, and therefore the polynomial Q(V) are given. 
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Assume also that the poles of A(D) are known. An explicit ex¬ 
pression for A(D) is determined and conditions deduced that it 
will be physically realizable for the case when the output has 
finite inertia. More precisely, let the poles of A(D) be — A l5 
— /U A /; , and let 


If b Q = 1, eqn(15) reduces to 


where 


A(D) = 


\k, r -1 i 


( 16 ) 


g(V)==£/ 0 + t/1V -1-hc/ / .V" = |} 0 + b 1 E H- \-b k E (13) 

Then 

C(V) 


A C (E)-- 




1 


= (exp£A,.) 


6(V) t 

IKv+O 

i 


/(/)=Ae- ; -e(-0//=log(/ + l)2(-0/H/+l) 

It is evident that the constant term in DA (D) vanishes if 

|fc,/(OI = 0 (17) 

This condition imposes a constraint on the possible values of 4 
and therefore of A,. This is now transformed into a form in which 
it can be interpreted and applied by a graphical method. 

If i k == o, the last row of j k,f(l) | becomes 


using cqn (5). 

In order to find the operator A ( D ) corresponding to A C (E) 
and to express the restrictions which must be imposed on this 
operator, use is made of determinants of the type 

1 /, Ff/j) 

1 l 2 ll-l k 2 ~ 2 F(l 2 ) 

i k k 2 -ir 2 m) 

This is denoted by \k, F(I)\. It is, in fact, a special type of 
alternant, that is, a determinant ||« (i || in which a is =/ 3 - (4) 
where j\ (/). f k (/) are different functions of I. 

Then it is well known that | k, I ' 1 " 1 1 is the product of the 
differences of pairs of the numbers 4,..., 4- If ex P A "4 1S 
denoted by . 1 it can easily be verified that the partial fraction 
expansion of A,. (E) can be expressed in the form 


1 0 0-0 /(0) 


Subtracting this row from the other rows gives 

i fc , / (i)i=(_) fc -u 1 / 2 -4. 1 ik-i,{/(o-/(o)}/;i 

so that eqn (17) reduces to | k - \,f 2 (/) | = 0 where 


/i(0=[/(0-/(°)]/i 

More generally, if 4 = 4-i = - = 4 -m = 0, it is found that 
eqn (17) is equivalent to 

\k-s,f s (l)\ = 0 ( 18 ) 


where 


/.(0= p 


/(o-/(o)-r(o)- 


(5-1)! 


0) (19) 


■]' 


AJE) = A 


V —) fc - 1 14% Q (— / )/(V + 01 


Ifc, /' 


7c-l. 


+ ^li 


-1 


(14) 


so that, using cqn (9), 


A (£>)=/! 


1 , ,/c - 1 


r 2 +i 

l(-) 

\ 

Z(D-hA) 

1 \k, l k ~ l \ 


-1 
(15) 

Now if the degree of the denominator exceeds that of the 

numerator by at least two, the constant terms in the part a 
«r A m of DA (D) must both be 

zero. Since e A — 1 1(1 T 1) an d 

| k, F x (/) + F 2 (/) | = | k, F 2 (/) | + I k, (0 I 

the first of these terms is 

T-)*- 1 ife,Q(-n/( i+ H lj./.. 


/i 


i/t, / t_ 


-i 


expressed as a^fraction 

no constant term. 1 his resuu lb unit , • nn otens It 

type of system prescribed, the °“K ie < ^tput at any sampling 
follows from this that the vahi P earlier sampling 

instant depends only on the value oi me f 
instants and not on its value at the same instan . 


It is now shown how this equation can be solved graphi¬ 
cally. In the simple case in which k- s = 2, eqn (18) reduces to 

/.(ii)=/.(W < 20) 

Corresponding values of 4 and /, can clearly be read off from a 
graph o(f s (/), as shown in Figure 3. 
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We consider now the case where k — s — 3, when eqn (17) 
takes the form 


1 h L(h) 
i h Lih) 
i h LAs) 


=o 


( 21 ) 


This shows that the points on the graph of f s (/) for which / takes 
the values / l5 l 2 and / 3 must be collinear, as in Figure 4 . It is 
evident that if two of these values are given, the third can be 
determined immediately. 



Figure 4 



In practice the range of values of / that may have to be 
considered is such that it is preferable to plot f s (/) against X 
rather than /. When k — s = 2 corresponding values X ± and X 2 
still give equal ordinates on the graph, but if k — s = 3 corre¬ 
sponding points on the curve now lie on an exponential curve 
of the type y — y 0 = e A ~*o 9 as shown in Figure 5. However, the 
shape of this curve is independent of y 0 and X 0 so that X z can be 
found immediately from X 1 and X 2 by means of a template cut 
in this shape. 

The cases in which k — s is equal to two or three should 
cover the majority of practical applications, but the graphical 
method used can be extended to higher values without undue 


complication. Thus if k — s = 4, points on the graph of f s (/) 
for related values of / lie on a parabola or on a curve with 
equation y = a e 2A + b e A + c, depending on the choice of 
abscissa. In either case, if three values are given the parabola 
or exponential curve can be plotted and the fourth value deter¬ 
mined by its intersection with the original graph. 

Applications 

The distribution and indeed the existence of sets of related 
real values of / or A depend on the shape of the graph of f s (l). 
This is now examined in one or two special cases. 

It is found from eqns (13) and (19) that 

JA) ~ r +i (/+1) i s 


+ 


3 

O’i +^"^0 

I s ' 1 


3 11 

a 2+-2 ai ^~6~ a ° 


r~ 2 


+ ••• 


( 22 ) 


the last term in this series being a multiple of 1//. Let the degree 
of Q (V) be h . It has been assumed so far that h = A% where k is 
the degree of the denominator of A (D). This is not necessarily 
the case. In fact, although h can never exceed k, there is no 
physical reason why h should not be less than k. We shall 
consider the case of staleness weighting, in which 

Q(V) = (l-aE- 1 ) , ' = (l—a)"(1 +£V)" (23) 


where /? = oc/( 1 — tx) and ix, the staleness weighting constant, 
is a real number, which for stability must lie between + 1. The 
important special case in which the system has a finite settling 
time is given by ot = 0. 

The potentially most useful case for detailed discussion is 
probably that in which s = 2 and k > 4, that is, a system in 
which A ( D) has a double pole at the origin and two or more 
non-zero poles. Now it is found in this case that if h < 4,f 8 (J) 
is monotonic, so that eqns (20) and (21) have no real solutions. 
We shall therefore take Q (/) = (1 — <%) 4 (1 + ft /) 4 , in which 
case eqn (22) becomes 


/ 2 (/) Ad-/?/) 4 1 2 +4/? 

(l — a) 4 1 3 (/+1) / 2 l 


(24) 


This function was tabulated for five different values of It using 
an electronic computer (see Table 1 and Figure 6). 

It is apparent that when ft = 0, f 2 (/) is again monotonic, 
from which can be inferred the interesting result that it is not 
possible to achieve a finite settling time with a system of this 
type. With a non-zero value of /?, however, eqns (20) and (21) 
become soluble. 

As an example, if k = 4, oc = ^ (or ft = •£) and X x =- 3, it is 
found from Table 1 that X 2 = 4*542. The subsequent calculation 
of A (D) is simplified somewhat if the following formulae, 
derived without difficulty from eqns (16) and (18) in the case 
when s — 2, are used. 


A (D) = 


{-) k A 

Jfc o 4f(0 

’ l 2 D(D + a) 

1 

k-2, l k ~ 3 1 


/ ( 0 ) -j-nr 


1 + '• 


(25) 
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H is found that 


4-37 D 2 +11 -42 D + 2-86 
A = D 2 (D + 3) (D + 4-54) 


(27) 


Thus if the plant operator is 1 ID (D + 3), in order to satisfy the 
specified conditions the compensator operator would have to be 

4-37 D 2 + 11 -42 D +2-86 

^ d+4 . 54) 

Alternatively, if the plant is a pure double integrator, with 
operator 1/D", a suitable compensator operator could be 

4-37 D 2 +1 1-42 D+2-86 
.(D + 3)(D+4-54) 

To take a more complicated example, with k = 5, s = 2 and 
„ = i (or /i 1) let 2, - 4 and 2 2 = 3. Then it is found from 
the graph, by using an exponential template as in Figure 5, 
that l, 1-3. If required, a more accurate value could be 
obtained by inverse interpolation. Substitution in eqn (25) an 
simplification gives 

1-01 P 3 +7-60£> 2 + 6- 92P + l’44 

A (D) = .d^d+4)TdTW+1-30) 


Thus for a plant whose operator is 

D + 2 

D (D + 4) (D + 3) 

the compensator operator to satisfy the given specification 
would be 

1-01 D 3 + 7-60 D 2 4-6-92 D +1'44 

- 'dJd+2)(D+TW~ 


Conclusion 

The theory given in .hie paper hae been atom »>™^a 

practicable method for solving the pro em taible 

in a wide range of cases. Basically, al tha . * "qurredUsit.Me 
, „ r r in fart the time taken to prepare uic 

of the function /„(0^ In fact, ]ess than an hoU r 

programme and to produce iaoi 


Table 1. Values off 2 (/)/(! — a) 4 when Q(V) = (1 — *E -1 ) 4 


P = 0 p = 0-25 p= 0-5 p= 0-75 p = 1 

a = 0 a = 0-2 a = 0-33 x = 0-43 a = 0-5 


0-0 0-00000 
0-2 0-22140 
0-4 0-49182 
0-6 0-82211 
0-8 1-2255 

1-0 1-7182 
1-2 2-3201 
1-4 3-0552 
1-6 3-9530 

1- 8 5-0496 

2- 0 6-3890 
2-2 8-0250 
2-4 10-023 
2-6 12-463 

2- 8 15-444 

3- 0 19-085 
3-2 23-532 
3-4 28-964 
3-6 35-598 

3- 8 43-701 

4- 0 53-598 


4*2 

65-686 

4-4 

80-450 

4*6 

98-484 

4-8 

120-51 

5-0 

147-41 

5*2 

180-27 

5*4 

220-40 

5*6 

269-42 

5*8 

329-29 

6*0 

402-42 

6*2 

491-74 

6*4 

600-84 

6*6 

734-09 

6*8 

896-84 

7*0 

1095-6 

7*2 

1338-4 

7*4 

1634-9 

7*6 

1997-1 

7*8 

2439-6 

8-0 

2979-9 

8*2 

3639-9 

8*4 

4446-0 

8*6 

5430-6 

8*8 

6633-2 

9*0 

8102-0 

9*2 

9896-1 

9*4 

12087 

9*6 

14763 

9*8 

18032 

10*0 

22025 

10*2 

26902 

10*4 

32858 

10*6 

40133 

108 

49019 

11*0 

59873 

11*2 

73129 

11*4 

89320 

11*6 

109096 

11*8 

133251 

12*0 

162753 


1-8333 

1-4624 

1-1695 

0-9376 

0-7534 

0-6067 

0-4896 

0-3959 

0-3206 

0-2601 

0-2113 

0-1718 

0-1399 

0-1140 

0-0929 

0-0758 

0-0619 

0-0506 

0-0413 

0-0338 

0-0276 

0-0226 

0-0184 

0-0151 

0-0123 

0-0101 

0-0082 

0-0067 

0-0055 

0-0045 

0-0037 

0-0030 

0-0024 

0-0020 

0-0016 

0-0013 

0-0011 

0-0009 

0-0007 

0-0006 

0-0005 

0-0004 

0-0003 

0-0002 

0-0002 

0-0001 

0-0001 

0-0001 

0-0001 


3-7083 6-3333 

2-9059 4-8446 

2-2825 3-7110 

1-7974 2-8490 

1-4193 2-1944 

1-1239 1-6982 

0-8926 1-3227 

0-7111 1-0393 

0-5684 0-8261 

0-4558 0-6663 

0-3669 0-5473 

0-2964 0-4594 

0-2405 0-3952 

0-1961 0-3491 

0-1607 0-3168 

0-1325 0-2950 

0-1101 0-2814 

0-0922 0-2739 

0-0779 0-2711 

0-0665 0-2720 

0-0575 0-2757 

0-0503 0-2815 

0-0447 0-2889 

0-0403 0-2976 

0-0369 0-3072 

0-0342 0-3175 

0-0323 0-3284 

0-0308 0-3398 

0-0297 0-3514 

0-0290 0-3633 

0-0286 0-3753 

0-0284 0-3975 

0-0284 0-3998 

0-0285 0-4122 

0-0288 0-4246 

0-0292 0-4371 

0-0296 0-4495 

0-0301 0-4621 

0-0306 0-4746 

0-0312 0-4871 

0-0319 0-4996 

0-0325 0-5121 

0-0332 0-5247 

0-0339 0-5372 

0-0346 0-5497 

0-0353 0-5623 

0-0361 0-5748 

0-0368 0-5873 

0-0376 0-5998 

0-0383 0-6123 

0-0391 0-6249 

0-0399 0-6374 

0-0406 0-6499 

0-0414 0-6624 

0-0422 0-6749 

0-0429 0-6874 

0-0437 0-6999 

0-0445 0-7124 

0-0453 0-7249 

0-0461 0-7374 

0-0468 0-7499 


9-7083 13-8333 

7-2223 9-9860 

5-3731 7-1990 

4-0069 5-2110 

3-0060 3-8225 

2-2807 2-8814 

1-7624 2-2726 

1-3994 1-9090 

1-1523 1-7250 

0-9913 1-6714 

0-8943 1-7116 

0-8444 1-8185 

0-8291 1-9720 

0-8389 2-1573 

0-8670 2-3639 

0-9081 2-5839 

0-9585 2-8120 

1-0153 3-0443 

1-0766 3-2781 

1-1408 3-5117 

1-2069 3-7440 

1-2741 3-9742 

1-3419 4-2021 

1-4099 4-4273 

1-4779 4-6500 

1-5457 4-8701 

1-6132 5-0879 

1-6804 5-3036 

1-7473 5-5173 

1-8138 5-7292 

1-8799 5-9395 

1- 9457 6-1484 

2- 0112 6-3560 

2-0764 6-5626 

2-1414 6-7683 

2-2061 6-9731 

2-2707 7-1772 

2-3351 7-3807 

2-3993 7-5837 

2-4634 7-7862 

2-5273 7-9884 

2-5912 8-1902 

2-6550 8-3917 

2-7187 8-5930 

2-7823 8-7941 

2-8459 8-9951 

2-9095 9-1959 

2- 9730 9-3965 

3- 0365 9-5971 

3-0999 9-7975 

3-1633 9-9979 

3-2267 10-1983 

3-2901 10-3985 

3-3534 10-5988 

3-4168 10-7990 

3-4801 10-9991 

3-5435 1M993 

3-6068 11-3994 

3-6701 11-5995 

3-7334 11-7995 

3-7967 11-9996 


169 













B. M. BROWN 


and similar tables using different values of the parameters 5 and 
h could be obtained equally easily. Alternatively, if no electronic 
computer were available the following reduction formulae 
could be used: 

//(s+l, h, /)=/ (s, h, l) -/<">(/)/s! (29) 

f(s, h+1, /) = -plf(s - 1, h, l)+f (s, h, l) (30) 

where f(s, h, /) =/ s (/) and Q (V) = (1 - « E^) h . Another 
possibility, which might well give a better response, is to choose 
a different form for Q (V), such as 1 ± oc h E~ h . 

The method can be extended to the case where the degree 
of the denominator exceeds that of the numerator by three. 
Possible values of A { must now satisfy the two conditions: 

\k-s,f s (l)l=0, \k — s,F s (l)\ =0 

where F s (l) is obtained from Xf(l) in the same way as f s (l) is 
obtained from /(/). 

A particular advantage of the graphical method is that it is 
possible to see from the general shape of the graph of f s (/) 
whether the specification can be fulfilled at all with a compen¬ 
sator of a given type. This general shape can often be inferred 
without actual computation of f s (/) by examining its asymptotic 
behaviour. 

Consideration has been confined so far to systems in which 
all the poles of A ( D ) are real. The theory still holds, of course, 
when some of these poles are complex, and it seems possible 


that a graphical method could be found for finding complex 
values satisfying eqns (20) and (21), if there should be a practical 
requirement for this. 

The author gratefully acknowledges the help given by Professor 
J. R. Lakey and Mr. J. Flower of the Department of Nuclear 
Science and Technology of the Royal Naval College , in the 
preparation of Table 1. 
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DISCUSSION 


Ya. Z. Tsypkin, Institute oj Automation and Telemechanics, Moscow 
1—53, Kalachevskaya 15-A, U.S.S.R. 

Professor Brown’s interesting paper examines the problem of synthesis 
of sampled-data systems with continuous compensation. Very few 
studies have been made of this problem. 

The difficulties involved in solving it stem from the fact that for the 
continuous part {Figure A) 


Wf(q) 


W 2 *(q) 


W(q) 

Figure A 




R. E. Kalman, Research Institute for Advanced Studies, 7212 Bellona 
Ave., Baltimore 12, MD., U.S.A. 

The problem discussed in this paper is an interesting one in the frame¬ 
work of early sampled-data theory. But today transform methods are 
seldom used for such problems because the state-variable approach is 
much simpler. And today one does not design by trial and error because 
optimal design is easier to explain and to compute. In the light of 
modern control theory, the solution of the problem becomes trivial. 

This solution, as is well known 1 , consists of two parts: (1) Deter¬ 
mine the optimal linear control law K{X) as a function of the state 
variables X (Reference 2, below); (2) Determine the optimal estimates 

X of the state variables X by a linear Wiener-Kalman filter 3 . 

These two steps may be carried out completely independently of 
each other 1 , and therefore either or both may be solved in the con¬ 
tinuous time or sampled-data (discrete-time) sense. In any case, the 
optimal control law is K(X ) 1 . 


(g 


= pT; T is the repetition period) 
But, as shown 1 , 


W*(q) = 


'l 

o 


W?{q,-X)-W*iq,X)dX 


( 1 ) 


It seems that this correlation (1) will make it possible to solve the 
problem under consideration by the same method as is used in the 
synthesis of pulse-correction systems. I should like to draw Professor 
Brown’s attention to this possibility. 

Reference 


1 oa^ IN ’ YA Theor y of Linear Sampled-data Systems. Moscow, 
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Following Dr. R. E. Kalman’s remarks that there is a need for the 
optimal control theory approach to this problem of the design of a 
continuous compensator for a feedback sampled-data linear control 
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system, as opposed to the two methods suggested here by Brown and 
Tsypkin, 1 would like to state that I have already successfully carried 
out such a design synthesis procedure under Dr. J. B. Knowles at the 
College of Science and Technology, Manchester, England. 

The object was to obtain the optimum continuous passive network 
of any discrete compensator. The criterion of synthesis used was that 


of minimum integral error squared. Satisfactory closed forms of the 
error function were obtained, and were then applied to a digital com¬ 
puter for the optimum constrained solutions of all of the independent 
variables of the continuous passive network compensator. 

Full details of this optimal design synthesis of a continuous com¬ 
pensator for a sampled-data system are soon to be published. 
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Summary 

Non-linear pulse control systems which consist of a non-linear 
element and a linear pulse part (LP) are considered. Using the ideas 
of V. M. Popov, the sufficient criterion of absolute stability is estab¬ 
lished from the frequency characteristic of the linear pulse part. 

A class of non-linear pulse control systems is defined for which 
this criterion of stability is also necessary. The applicability of the 
criterion of absolute stability is shown for non-linear pulse control 
systems in which the characteristic of the non-linear element is an 
explicit time function. 

It is established in which cases the absence of periodic con¬ 
ditions guarantees the absolute stability of these non-linear pulse 
control systems. Methods are given for estimating indices of the 
quality of a process: the degree of stability and the overall square 
deviation. The stability and quality of non-linear pulse control sys¬ 
tems can be investigated comparatively simply by concepts and 
representations of the theory of linear pulse systems. 

Sommaire 

On considere des systemes asservis echantillonnes nonlineaires com- 
prenant une partie nonlineaire et une partie 6chantillonnee lineaire. A 
partir des idees de V. M. Popov, on etablit la condition suffisante de 
stability absolue a partir des caracteristiques frequencielles de la partie 
echantillonnee lineaire. 

On definit une classe de systemes echantillones nonlineaires pour 
lesquels ce critere de stabilite est egalement necessaire. On montre 
l’applicabilite du critere de stabilite absolue a ceux dans lesquels 
l’el6ment nonlineaire est une fonction explicite du temps. 

On etablit dans quels cas Fabsence de conditions periodiques 
garantit la stabilite absolue de ces systemes. On donne des methodes 
pour evaluer les indices de qualite d’un processus: le degre de stabilite 
et l’ecart quadratique global. La stabilite et la qualite de ces systemes 
peuvent etre etudiees assez simplement en partant de la theorie des 
systemes echantillonnes lineaires. 

Zusammenfassung 

Nichtlineare Abtastregelsysteme, welche aus einem nichtlinearen 
Element und einem linearen Impulsteil (LP) bestehen, werden hier be- 
sprochen. Das ausreichende Kriterium fur absolute Stabilitat wird 
nach V. M. Popov auf Grund der Frequenzcharakteristik des linearen 
Impulsteiles aufgestellt. 

Es wird eine Gruppe nichtlinearer Abtastsysteme definiert, fur 
die dieses Stabilitatskriterium ebenfalls notwendig ist. Die Anwend- 
barkeit des Kriteriums der absoluten Stabilitat wird fur nichtlineare 
Abtastregelsysteme gezeigt, bei denen die Charakteristik des nicht¬ 
linearen Elementes eine explizite Zeitfunktion ist. 

In bestimmten Fallen garantiert das Fehlen periodischer JBe- 
dingungen die absolute Stabilitat derartiger Regelsysteme. Es werden 
Methoden fur die Abschatzung der Qualitatskennzeichen eines Pro- 
zesses angegeben: der Stabilitatsgrad und die gesamte quadratische 
Abweichung. Stabilitat und Giite solcher Systeme konnen verhaltnis- 
maBig einfach mil Hilfe von Begriffen und Darstellungsweisen aus der 
Theorie nichtlinearer Abtastsysteme untersucht werden. 


Introduction 

The theory of linear pulse control systems has attained a high 
level of development and the main problems in the analysis and 
synthesis of such systems can be solved. However, with regard 
to non-linear pulse control systems, the theory is still in its 
initial stage. Up to the present time non-linear theory has been 
confined mostly to the investigation of periodic regimes. 
Yet periodic regimes are not operational conditions and the 
important problem still remains to ensure the stability of non¬ 
linear pulse control systems and to assess the ‘quality’ of stable 
processes. Attempts to employ the methods of investigating 
periodic regimes for estimating stability when the required 
periodic regimes are no longer present are often unjustified 
since the absence of a particular type of periodic regime is 
no guarantee that other forms of periodic or almost-periodic 
regimes are not present. 

For solution of the stability problem it was quite natural 
to try to employ the ideas of Liapunov’s second method which 
is widely used in the theory of continuous systems, in extend¬ 
ing them to difference equations 1 ” 5 . 

However, such an approach involves difficulties associated 
with the need to transform the equations of non-linear pulse 
systeme into their normal form, the arbitrariness of the selec¬ 
tion of Liapunov functions and the impossibility of establishing 
any general properties of non-linear pulse control systems. 

The approach to the problem in this paper is based on an 
idea which Popov 6 * 7 used in the investigation of non-linear 
continuous control systems. The distinctive feature of this 
approach is that it is closely associated with such physical 
concepts as the frequency and transient responses, and it pro¬ 
vides the widest sufficient conditions of stability which can be 
obtained by all the Liapunov functions of the quadratic type. 
This approach greatly simplifies an assessment of the quality 
of processes in non-linear pulse control systems. It is possible 
to establish when the absence of periodic regimes guarantees 
stability and, finally, use may be made of methods similar 
to those employed in the investigation of linear pulse systems. 

Statement of the Problem 

A block diagram of a non-linear pulse control system is 
shown in Figure 1. It consists of a non-linear element in 
series with a linear impulse part LP which is an open linear 



Figure 1 
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pulse loop. The linear impulse part incorporates a pulse element 
for amplitide modulation of arbitrarily shaped pulses and 
a continuous part. 

Let us suppose that the characteristic <D (x) of the non-linear 
element satisfies the following conditions {Figure 2 ): 


(a) 

<6(0)=0 

(b) 


X 

(c) 

lim O(x)=±0 


X-+ ± 00 


which correspond to the fact that this characteristic belongs to 
the sector (0, k). 

The main problem is to determine the stability of the systems 
which are to be considered for any initial deviations and to 
determine the quality of behaviour in stable systems. Stability 
of this kind which is independent of the particular shape of the 
characteristic of the non-linear element and which satisfies the 
general conditions (1), is called generally absolute stability 8 . 



Equations of Non-linear Pulse control Systems 

Let one suppose that the continuous part of the linear pulse 
part LP receives perturbations in the form of initial conditions 
with n = 0. One puts f[n] for the response of this continuous part 
to partial conditions and applies it to the input of the non-linear 
pulse system {Figure I). If the continuous part, and therefore 
the linear pulse part, is stable, then 

lim /[n]=0 (2) 

n~+ oo 

The equation of the pulse control system with, respect to the 
error x [n] can take either of two forms. 

(/) With respect to original lattice functions: 

x [n] =/ [n] - z w [n - m] $(x [m]) (3) 

m = 0 

(//') With respect to their transforms: 

X*(q) = F*(q)-W*(q)D{<S>(x[n])} (4) 

Here 9 

Z*(4) = £{z[n]}=Ze-*"z|>] (5) 

n = 0 

is the discrete Laplace transform {D transformation) \q = a -\- jdj 

is a parameter of transformation; cb = coT is the relative 

frequency; T is the repetition interval; 

W*(q)=D{w[n]} (6) 


is the transfer function of the linear pulse part; 

w[n] = vv(r) 7 = „ (7) 

is the impulse characteristic of the linear pulse part; x[n], /[ft] 
are the lattice functions, which correspond to the error and the 
input: X*{q), F*{q) are their transforms and, finally, O (a) is 
the characteristic of the non-linear element. 

For a stable linear pulse part 

lim w[ft] = 0 (8) 

co 

This implies that the corresponding transfer function W*(q ) 
has no poles in the right-hand half-band Re#>0, —n< Im q<7t. 


The Sufficient Condition of Absolute Stability 


A pulse control system is absolutely stable relative to any 
perturbation / [ft] which satisfies the condition (2) if 


lim x[n]=0 


(9) 


In order to establish the fact of absolute stability, one estimates 
the solutions x [n\ of the equation with respect to the original 
functions. 

By analogy with the ideas of Popov 6 * 7 , the auxiliary func¬ 
tions are introduced 


<Pn [>] = 


and 


where 


fa (*[«]) 
to 


0 <n<N 
n<0,n> N 


il/ N [_n] = x N [_n]- — (p N [n] 


( 10 ) 

( 11 ) 

( 12 ) 


Xjv [h] =/[”]- £ W [n - m] cp N [m] 

m — 0 

It is obvious that for 0 < n < N 

X N [ft] = X [ft] 

where x [n] is the solution of eqn (3). 

Now the following expression is formed 


Pn= E Pjv ln]i// N [h] (13) 

n = 0 

which, having regard to (10) and (11), is equal to 

Pn = (x [»])X W - ■-jr-fl* 2 (x [n])^ (14) 

According to the Liapunov-Parseval equality 9 eqn (13) can also 
be represented as 


where 


1 [ K 
P * = 2^L 




and by virtue of (11) and (12) 

W N (jco)=D {ip N [n]} q=jm =F*(jd5)-(w*(ja5) +-|r) <3E> * 


(15) 

(16) 

(Jw) 

(17) 
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These spectral functions exist if conditions (10) and (8) are 
fulfilled. 

Substituting (16) and (17) into (15) and after simple trans¬ 
formations one gets 


Pn~~ ~~ 


2 n 


VRen*(jffi)4>J(jcS)- 


F* (jco) 


+ 7 


71 F* (ja>) 
-* Re n* (jco) 


dco 


where 


Re TL* (jco) = Re W* (jco) + — > 0 


2 v Re n* (Joj)\ 


1 


The function 


n*(ja>)=W*(j(8) + 


1 


dco 

(18) 

(19) 


which is the slope of a straight line passing through the origin 
and the point of the non-linear characteristic for a specified 
value of x. The maximum S max and the minimum S min static 
gains are determined by the rays of a sector which is tangential 
to the characteristic (Figure 3). A non-linear pulse control system 
in which the non-linear element is replaced by a linear element 
with some fixed gain, k, is said to be a linearized pulse control 
system. For a linearized pulse system to be stable, by analogy 
with the Nyquist criterion 7 , it is necessary and sufficient that the 
frequency characteristic of the linear pulse part LP should 
not embrace the point — 1/kJO. It will be said that a linearized 
system is obviously stable if the frequency characteristic of the 
linear pulse part does not intersect the straight line — l//c. 
Then, according to the condition of stability (23), the fre¬ 
quency criterion of absolute stability of a non-linear pulse 
control system can be formulated in the following way. 


which plays the main role, is called the analogue of the Popov’s 
function. 

Since the first integral in (18) is negative, by discarding it, 
one obtains the inequality 


Pn — 


_1_ f 71 

3 71 J -» 


iR*o~a)i 2 

Re n* (jco) 


dco = C 


( 20 ) 


By virtue of (19) the quantity C is positive: it is independent of N. 

Substituting into the left-hand side of (20) the value of qn 
from (14) one obtains 

<*>(* [n])' 


£ 0(x[n])x[n]( 1 


k x [ft] 


<C 


( 21 ) 


According to the condition (1 a), the sum on the left-hand side 
of (21) is positive, moreover it is limited. The series which is 
formed from this sum as TV—> oo, therefore converges. Using 
the known theorem of the convergence of series with positive 
terms, one concludes that 


lim <D (x [ft]) x [ft] 

n~* oo 



(DQj>]) \ 
k x [ft] / 


= 0 


Hence, by virtue of the conditions (1), it follows that 

limx[ft]=0 (22) 

n~> oo 

Thus a pulse control system which has a stable pulse linear 
part and a non-linear characteristic <D (x) and which satisfies 
the conditions (1), will be absolutely stable if the real part of the 
analogue of Popov’s function is positive, i.e. if 

Ren*(jS)=ReJr H O'a>)+T>0 (23) 

The condition of stability (23) determines the magnitude of 
the sector (0, k) which includes the non-linear characteristic 
CD (x) for which the pulse system is absolutely stable. This 
condition is sufficient. 


Frequency Criteria of Absolute Stability 


To formulate the criteria of stability of a pulse control 
system one introduces the concept of a static gain of the non¬ 
linear element 


S(x) = 


d>(x) 


(24) 



A non-linear pulse control system with its characteristic be¬ 
longing to the sector (0, k ), will be absolutely stable if the 
linearized pulse system corresponding to it is obviously stable 
or if the frequency characteristic W*(joo) of the linear pulse 
part does not intersect the straight line — l//c (Figure 4). 

The greatest value k = k Q which determines the span of 
the sector in which the non-linear characteristic is located, is 
determined by drawing the vertical tangent to W*(joo). The 
difference k — *S raax characterizes the margin of stability. 

The stability criterion of a pulse control system can also 
be formulated with reference to the frequency characteristic 
K*(jcd) of a closed linearized pulse control system. Selecting 
/; = & 0 /2; then 

^ w*a<o ) 

K*Uc5)=— t - (25) 

1 +fW*(jco) 
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According to the usual constructions of the frequency 
characteristic of a closed loop from the frequency characteristic 
of an open loop 9 , for an obviously stable linearized pulse control 
system if k = kj 2, one has 

\K*(jco)\<l (26) 

Thus a non-linear pulse control system with its characteristic 
belonging to the sector (0, k 0 ) will be absolutely stable if the fre¬ 
quency characteristic of the closed linearized pulse control system 
7<T*(a)) with gain Ar 0 /2 does not exceed unity in absolute value. 

One notes that the frequency criteria are also applicable in 
those cases when the continuous part contains delay elements 
or elements with distributed constants. 

The frequency criteria of absolute stability can also be 
expressed in analytic form. The first criterion is closely related 
to the problem of Karateodory, whilst the second criterion is 
closely associated with Shur’s problem in the theory of analytic 
functions 10 . 

The analytic form of the criteria is considered in a special 
paper. One will not consider it here as, more over, the use of 
frequency criteria is the simplest way of elucidating various 
general properties of non-linear pulse control systems. 



Generalization of the Stability Criteria 

Non-linear pulse control system which contain a stable 
linear pulse part have been considered above. Now suppose that 
the linear pulse part is neutral or unstable. This implies that 
its transfer function W*(q) has poles on the imaginary axis, and 
in particular, at the origin or in the right-hand half-band 
Re q > 0, — n < Im q < rc 9 . Since the determined sufficient 
conditions must hold for any non-linear characteristic which 
belongs to the sector (0, k Q ), they must also hold for a linear 
characteristic which belongs to this sector. But for sufficiently 
small gains z of this linear characteristic, a closed pulse control 
system will behave like an open pulse control system corre¬ 
sponding to the linear pulse part, i.e. it will be neutral or un¬ 
stable. Therefore, for instances of a neutral or unstable linear 
pulse part it is necessary to impose additional limitations on 
the minimum static gain S m - m . Let us elucidate these limita¬ 
tions. Let us introduce proportional feedback with the coefficient 
z across the linear pulse part {Figure 6), one supposes that the 
structure of the linear pulse part is such that for a finite z < S m j n 
the closed linear pulse part is stable. The frequency criteria of 
stability are then applicable to this non-linear pulse control 
system, but the role of the frequency characteristic of the 




I I I 


frm 


x [n] 


-— o— 10 (x )| — 1 —““I 


(aj 




LP i i } 

,--r±i=t: 


(b) 


Figure 6 


linear pulse part W*(jco) will now be played by the frequency 
characteristic of the closed pulse control system, which is a new 
linear pulse part equal to 


W e *(jco) = 


wHm 

1-f rW*(jeo) 


(27) 


But the block diagram of a non-linear pulse control system 
[Figure 6(a) ] can easily be converted to the form of Figure 6(b) 
where f[n] is now the response of the closed linear pulse 
control system, and the non-linear characteristic is equal to 


d? (x) — zx (28) 

However, since this characteristic must satisfy the condi¬ 
tions (1), 

r <^N</ c 1 + Z = K (29) 

X 

i. C. ^min > 2 


Thus the formulation of the frequency criterion remains unchan¬ 
ged. Only the characteristic of the non-linear element must now 
belong to the sector (z, /c-fz), and the frequency characteristic 
of the linear pulse part W't(jcb) is determined by the expression 
(27). 

One notes that if the linear pulse part is neutral and its 
transfer function W*(q) has only one pole at the origin, whilst 
the rest of the poles have negative real parts, then z in eqn (27) 
can be arbitrarily small and for this case one has 

W* (joj) ■ (30) 

i.e. in this case there is no need to construct lY£(ja>) from 
W*(jco) on the basis of the relation (27). 

If the non-linear characteristic ® (x) at x > x° goes outside 
the limits of the sector (z, k + z), which is usually the case for 
non-linear characteristics of the saturation type, the frequency 
criterion of stability guarantees stability with deviations of the 
error not exceeding x°. 

The frequency criteria of stability also hold for those cases 
when the non-linear characteristic (or gain of the linear pulse 
part) is a function of time n f if <E> (x, n) for any n > « 0 satisfies 
the conditions (1), i.e. if it belongs to the sector (0, k Q ) or in 
the case of a neutral or unstable linear pulse part belongs to 
the sector (z, k 0 + z). 
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The Necessary and Sufficient Conditions of Absolute Stability for 
Some Non-linear Control Systems 

Frequency criteria of absolute stability determine the 
sufficient conditions of absolute stability. It is obvious that in 
those cases when these sufficient conditions of absolute stability 
coincide with the necessary and sufficient condition of stability 
of linearized pulse control systems, they also become necessary 
conditions of absolute stability. Let us define the class of non¬ 
linear pulse control systems for which the conditions of absolute 
stability are necessary and sufficient. This problem was first posed 
by Aizerman 11 , for continuous control systems, and slightly later 
by Letov 8 . The solution of this problem is of importance since 
it permits reduction of the investigation of the absolute stability 
of non-linear pulse control systems to the well-known investiga¬ 
tion of the stability of linear pulse control systems. 

It follows directly from the formulation of the frequency 
criterion that this class of non-linear pulse control systems 
includes those for which the obvious stability of linearized pulse 
control systems coincides with their stability. The frequency 
characteristics of these latter pulse control systems W*(j(F) [or 
Wfijcfi)] must have the form shown in Figure 7(a) and (b). 



(a) Figure 7 



The frequency criterion of absolute stability determines the 
necessary and sufficient conditions for all non-linear pulse con¬ 
trol systems of the first order (with amplitude- or pulse width- 
or time-modulation), and also for non-linear pulse control systems 
of any order whose frequency characteristic W*(jcd) has the 
largest real part in absolute value at the boundary frequency. 
It is worthwhile pointing out that for this class of system the 
absence of periodic conditions according to the improved 
method of harmonic balance 12 , testifies to their stability. For 
digital automatic systems, as shown elsewhere 13 , the deter¬ 
mination of periodic regimes with a relative frequency a> = n 
entails drawing a straight line with a slope — l/W*(ja)) in the 
plane of the non-linear characteristic (Figure 8) *. If the maximum 
real part W*(j(b) in absolute magnitude is attained for = n 
(which always occurs for first-order pulse control systems), the 
condition requiring the absence of periodic regimes with 
a relative frequency co = n coincides with the condition of 
absolute stability. 


Estimation of the Degree of Stability 

For the simplest estimate of the quality of the behaviour 
of a non-linear pulse control system, one will use the concept 
of degrees of stability which characterizes the process damping 
speed. 

* The author points out that in a previous paper 12 he has given an 
erroneous slope. 


For this purpose, instead of the auxiliary functions (10) and 
(11), the following functions are introduced. 


and 


<Pn [«] = 


4> (x [n]) e Sn 

0 


0 <n<N 
n<0,n>N 


M = X N [>] -~1-<Pn 0] 


(31) 

(32) 


where d > 0 is some constant quantity. 

Multiplying both sides of (12) by e 6n , there is obtained 


%W^=/[n]^"- I (33) 

m -0 

Remarking that according to the shift theorem 9 

D{z\n\e d X^=Z*{-5 + jl3) (34) 



and following the same discussion as in the establishment of the 
condition of absolute stability, the conclusion is reached that 

lim x!>]£> = 0 (35) 

if the real part of the analogue of the shifted function of 
Popov is positive, i.e. if 

Ren*(-S + jco)=ReW*(-d + jco)+-^>0 (36) 


As will be seen from eqn (35), the rate of damping is deter¬ 
mined here by the quantity 3. The determination of the condi¬ 
tions for which non-linear pulse control systems have a specified 
degree of stability, 3 0 , thus entails the use of the frequency 
criterion of stability and its application to the shifted frequency 

characteristic * 

W (-5 0 + jco) (37) 


or 


W*(-8 + jo5) = 


1 + zW*(— <5 0 + jo)~) 


for a fixed value <3 0 (Figure 9). 



(38) 
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Since the poles of the transfer function W*(— d + q) depend 
on d and with increase of d are shifted in the direction of the 
right-hand half-band, the greated value of <5 = <5 max is attained 
for a value 


Z 0 — *^m i n 


(39) 


which still ensures stability of a closed linear pulse part. Thus 
the increase of S is possible until the poles FF*(— d + q) are 
located at the origin or on the imaginary axis. With an increase 
of d the quantity k Q usually decreases, whilst z increases. There¬ 
fore, the less the difference S mSLX — S min , the more attainable is 
a large degree of stability. This estimate is also applicable to 
non-linear pulse control systems in which the characteristic of 
the non-linear element depends also on time. 


The Quadratic Estimate 


Another important estimate of the quality of behaviour is the 
overall quadratic estimate of the output of a non-linear element 


12 = I <I> 2 (*!>]) 


(40) 


n~0 


To determine the upper boundary of this estimate, one will 
avail oneself of Popov’s ideas 14 . Consider the inequality (21) for 
N — co, representing it in the form 


Px 


*[«] 


Since 


= ll ? 0 ^ (xW) U ( xW ) k 

1 


<C 


(41) 


<t ) W > S'max 


the inequality (41) can be strengthened and 

1 1 


S m , 


H <J> 2 (x[n])<C 


(42) 


n = 0 


Taking into account the notation of (40) and (20), from (42) 
one gets 


/,< 


8 71 


d» (43) 


Re n* (jco) 


where 


1 1 1 


Re n* (jw) = Re W*(jco)+j>j-f>0 


(44) 


Replacing ReTI*(./co) in eqn (43) by its maximum value, one 
finally gets 


l< 0 k S a 


1 


l2 -(k-S m J{k 0 -k) 87i 


|F*(jco)| 2 da5 (45) 


The right-hand side of inequality (44) contains an undetermined 
parameter k ; here (Figure 3) 

S mM <fe<k 0 ( 46 ) 

This is so selected that the coefficient is minimum for the inte¬ 
gral (45). It can be shown without difficulty that in this case 

2 k.Q S'max 


and therefore finally get 

k 2 0 S } 


k 0 + Sje 


U< 


2 q2 
max 


(k 0 — S max ) 2 7i 


\F*(joS )\ 2 dm 


(47) 

(48) 


But according to the Liapunov-Parseval 9 equality, 

% 00 

|F*0®)l 2 dfii=I/ 2 [n] (49) 

-7i n = 0 

Therefore eqn (48) can also be represented as 


_1_ 
271 


/ 2 <^i/ 2 w 

V^O °max7 n -0 


(50) 


It follows from (50) that the upper boundary of the overall qua¬ 
dratic estimate is determined by the sum of the squares of the 
discrete responses of the linear impulse part to the applied 
inputs. If the linear impulse part receives an input/j [n] which 
decreases with time, then 

n 

/[«]=£ w[n —m]/) [m] 

m = 0 

and this implies that 9 


F* (jco) = D {/ [n]} 9= , s = 5 { W* ( q ) F\ 

The computation of the right-hand sides of (48) or (50) is carried 
out analytically or graphically by known rules 9 . The upper 
boundary of the overall quadratic error is less, other things 
being equal, the greater the margin of stability k 0 — ^max. This 
estimate is also applicable when the characteristic of a non¬ 
linear element depends also on time. 


Conclusion 

This approach to the problem makes it comparatively simple 
by the concepts of the linear theory of pulse systems to determine 
the region of absolute stability of non-linear pulse control systems 
and to estimate indices of the quality of processes (the degree of 
stability and the overall quadratic estimate). The fact that the 
stability and estimates of indices of process quality are independ¬ 
ent of the actual shape of the characteristic of the non-linear 
element, provided only that this characteristic belongs to the 
specified sector, makes it possible to ensure values of estimates 
of the indices of quality for variation of the characteristic of the 
non-linear element or of the parameters of the linear pulse part 
which also lead to a change in the boundaries of the sector 
(z s k + z). In some cases it is therefore no longer necessary to 
use special additional self-adjusting circuits which complicate 
non-linear pulse control systems. 

In this connection it is extremely important to determine the 
structure of non-linear pulse control systems, the sensitivity 
of which is low in relation to variations of the non-linear 
characteristic and to the parameters of the linear part. For 
this purpose use may be made of the results of investigations 
into the sensitivity of linear pulse control systems. 

Generalization of the method of investigating non-linear 
pulse control systems to pulse control systems which contain 
a linear pulse part with time-variable parameters, and several 
non-linear elements, widens the range of problems which can be 
solved and, in particular, makes it possible to investigate non¬ 
linear pulse control systems in wich pulse-width, pulse-phase 
and pulse-frequency modulation is provided. 
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DISCUSSION 


G. P. Szego, Research Institute of Advanced Studies, 7212, Bellona 

Ave., Baltimore 12, MD., U.S.A. 

This paper is another important contribution that Professor Tsypkin 
has made to the theory of sampled-data systems. 

My comments range from the use of Liapunov functions for the 
solution of the problem treated in the first part of this paper, to the 
problem of absolute stability of sampled-data control systems. In the 
paper of Kalman 1 , published at the beginning of this year and in which 
he extends a result due to Yakubovich 2 , the connections between the 
second method of Liapunov and the method of Popov are shown for 
continuous systems, I have been concerned with the analogous prob¬ 
lem for discrete systems. 

I shall very briefly state the results obtained and outline the 
method used. 

Given a sampled-data control system 2 

x t+l =Ax t -acp(c t,) (1) 


For the existence of real y, q and H — TV satisfying (5) it is necessary 
and sufficient that 

L+2Reb'(Iz-A)~ 1 a>0 (6) 

is satisfied for all real z = e i(ti , co real. 

This condition may be rewritten as 

L+ReW(z)>0 (7) 

K 

where W(z) = 2 b (Iz — A ) _1 a is the open loop transfer function of 
the linear part of 2. This condition is (23) of the paper. By the hypo¬ 
thesis of complete observability it follows that (4) is negative definite 
along (1). Thus (1) is absolutely stable if (6) is satisfied. 

(b) Let us proceed further along this line. Assume that cp has 
continuous and bounded derivative and define 


<y t =2 b'x t t = 0 , 1 , ... 

<p(0) = 0, 0<ctcp(a)<<j 2 k (2) 

Assume A stable, with at the most one eigenvalue equal to unity, 
the linear part of 2 completely controllable and observable, and all 
linear systems of the class cp — am, 0 < m< k, asymptotically stable. 

(i a ) It can be proved 3 that the stability condition (23) of the paper 
is a sufficient condition for the existence of a quadratic Liapunov 
function 

v = x'Hx (3) 


as in Figure 2 of the paper. 

By using the Liapunov function 


( 3 ) 


v = x r Hx + 


(p(s)ds 


(9) 


one can show 4 that for (1) to be absolutely stable it is sufficient that 


which proves the absolute stability of 2. 

Consider the u-difference along the solutions of (1): 


where 


Av = — [yep + q' x~\ 2 — cp 



(I) H + A'HA — qq' 
(II) A'Ha — b — yq 

(III) L-a’Ha = y 2 


(y+iS)L + Re(y + /? 2 )^( z )-^l|(z-l)^(z)| 2 >0 

( 10 ) 

( 4 ) 1 

(y + t3) T + 2 iP a ' b -k o \p\{a'b?¥>0,y + p>0 

are satisfied for all z — e ia) , co real and some real y and /?. 

(5) (c) Under the same assumption which led to the condition (26) and 
the relaxed conditions on A, such as the sixth section of the paper, by 
using the Liapunov function (7) one can show 5 that the system (1) is 
absolutely stable if 
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ijj > 0 

2^a'b + (a-^~a-p>0J > 0 


( 11 ) 


are satisfied for all z — e i( °, co real and some realy and /?. 

Even if we have now some fairly good results on this problem the 
status of stability theory for sampled-data systems has still not reached 
a satisfactory stage as in the continuous case. Further work is needed 
and improvements can be achieved by using the new method of 
Popov 5 *' 5 and very likely by using some more sophisticated Liapunov 
functions. 

Professor Tsypkin mentioned in the paper the possibility of ex¬ 
pressing the basic frequency criteria of absolute stability in analytic 
form. Could he discuss this point in some detail? 
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E. I. Jury, Department of Electrical Engineering , University of Cali¬ 
fornia, Berkeley, California, U.S.A. 

This paper represents a useful and important contribution to the 
absolute stability of non-linear discrete (pulse) control systems. The 
main feature of the method of this paper is the extension of Popov’s 
approach (applied to the continuous case) to the discrete case, thus 
simplifying the underlying procedure for stability test and assessing 
the quality of control. 

While the method of this paper mainly provides a sufficient con¬ 
dition for stability, it also provides for certain non-linear discrete 
systems the necessary and sufficient condition as explained by the 
author. This extended feature, among others, provides an added advant¬ 
age on the use of Liapunov’s quadratic form. For this class of systems 
in which were obtained the necessary and sufficient condition, I might 
mention that Figure A of this discussion could be added to the forms 
indicated by Professor Tsypkin in Figure 7(a) and ( b ) of his paper. 



More than a decade ago, I showed in my Doctoral Thesis that such a 
convex locus could represent the limiting case when T (the sampling 
period) is very large compared with the linear system time constants. 
It would be interesting to prove analytically that for this limiting case, 
Tsypkin’s method yields the necessary and sufficient condition for 
stability. 

The class of system where the method yields a necessary and suf¬ 
ficient condition is rather limited (confined to first-order or the limiting 
case) and thus for the most encountered systems the method represents 
the sufficient condition. In view of the latter restriction it is of interest 
to discuss the problem of possible improvements on the sufficiency 


condition. This problem which was stimulated by private correspond¬ 
ence with the author, has been recently studied and certain results 
have been obtained which are briefly discussed in the following. 

In order to improve on the sufficiency condition, I find, in col¬ 
laboration with Mr. B. W. Lee, that this could only be done if some 
other restrictions on the non-linearity discussed in Tsypkin’s equation 

(1) is further imposed. This additional restriction limits the so-called 
(a.c.) gain* of the non-linear element to less than 1C. For such a class 
of systems the following theorem has been proposed and the proofs 
will be published in the near future. The stability theorem requires 
that the following relationship be satisfied on the unit circle, i.e. 

ReG*(z)[l+q(z-\)-]+E-^\(z-l)G*(z)\ 2 Z0 

where q is a non-negative number and Re represents the real part. 

Certain features of the stability criterion embodied in the above 
theorem can be briefly mentioned. 

(1) The application of the criterion requires knowledge only of 
the Z transform of the linear plant on the unit circle (z = exp .jco), 
which may be derived easily from the frequency response of the linear 
plant and the compensating network. This is a characteristic feature 
of results obtained by Popov’s method. 

(2) This criterion is directly applicable to systems with linear plants 
of any order. This follows since the criterion is stated in terms of the 
Z transform of the linear plant rather than the time-domain state 
variables of the system. 

(3) The criterion includes Tsypkin’s results as a special case, 
i.e. when q — 0. 

(4) For the limiting case when T —■> 0 (i.e. the discrete system 
approaches the continuous case) this criterion can be shown to coincide 
with Popov’s method. 

For the brevity of the discussion, we will not elaborate on the 
above points since these will be discussed in detail in a future publi¬ 
cation; however, at the conclusion one may note the following: 

A general comparison between Tsypkin’s criterion and the one 
presented above is somewhat difficult since the class of systems con¬ 
sidered by the former is more general. This is due to the extra restric¬ 
tion we have imposed on K'. However, for all cases where least con¬ 
servative results are obtained with q ~ 0, results obtained by either 
criterion may be evaluated on an identical basis. If q 4= 0, results are 
comparable only when the system is restricted to being a member of 
the class considered in the new criterion. This method yielded better 
results, for the examples worked out for this class of systems, than 
those obtained with the method mentioned in the paper. 

Reference 

Jury, E. I. and Lee, B. W. On the stability of a certain class of non¬ 
linear sampled-data systems. I.E.E.E. Transactions on Automatic 

Control. In the Press 

C. C. Li, Department of Electrical Engineering, University of Pitts¬ 
burgh, Pittsburgh, Pennsylvania, U.S.A. 

The author is commended for this excellent contribution to the non¬ 
linear pulse control system theory. A research project on determining 
the discrete system stability and minimizing the sum of quadratic 
error, via Liapunov’s second method, has been done by our group at 
the University of Pittsburgh. The author may be interested in noting 
R. P. O’Shea’s Sc. D. Thesis ‘On the Determination of Domain of 
Asymptotic Stability of a Class of Non-linear Discrete-Data Control 
Systems’, Department of Electrical Engineering, University of Pitts¬ 
burgh, 1962. 

Has the author actually applied this method to pulse frequency 
modulated control systems ? 

* The slope or derivative of the non-linearity. 
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Ching-shuan-Shue, Harbin Institute of Technology , China 

Up to the present time, theory of non-linear pulse control systems has 
dealt mostly with the investigation of periodic oscillations, which, 
however, are not the frequently occasioning conditions of non-linear 
pulse control systems. 

This interesting paper by Professor Tsypkin gives some new means 
of studying the non-linear pulse control systems. It offers a frequency 
criterion of absolute stability by the concepts of the linear theory of 
pulse systems and a method of estimation of indices of the quality 
of processes. 

However, there are two points in the article that are not very clear. 
If the parameters of the linear pulse part, or those of the non-linear 
element, are time-varying, for example, when they are periodic func¬ 
tions of time, will the frequency criteria of stability remain unchanged 
or would some corrections have to be made ? 

Moreover, this article considers the cases, where the lattice function 
f[n ] approaches zero as limit when n approaches infinity. Will the 
proposed method remain unchanged when 

lim /[«]== const. 

«-> 00 


Ya. Z. Tsypkin, in reply 


I am grateful to Dr. G. P. Szego, Professors E. I. Jury, C. C. Li and 
Ching-shuan Shue for their very interesting discussions and some not 
less interesting questions and remarks. 

Dr. Szego has pointed out the relationship between the criterion of 
absolute stability proposed with the second method of Liapunov and 
states that inequality (23) is sufficient for a quadratic function of 
Liapunov. This fact was well known to us. The further development 
of absolute stability criteria under the assumption of the additional 
limitation 


0 < 


dffQ) 

dcr 


<K 0 


is most interesting. On the other hand, I cannot agree with the opinion 
of Dr. Szego that the problem of absolute stability of non-linear pulse 
systems does not reach a satisfactory level as in the case of analogue 


systems. On the contrary, the solution of this problem for non-linear 
pulse systems has a more simple and accomplished form. As concerns 
the formulation of analytical conditions for absolute stability, only the 
preliminary results were obtained in this direction and they are un¬ 
fortunately given in a very complicated form. Professor Jury confirms 
that the criterion of absolute stability proposed, very often gives 
extremely hard conditions. Using the additional limitations 



these conditions can be eased. The inequality given by Professor Jury 
most probably coincides with the inequality proposed by Dr. Szego. 
It must be pointed out that these inequalities give the solution of the 
problem of absolute stability in contrast to the problem of our paper 
considered under the existence of two limitations. Though the solution 
of this problem has a more complicated form, it is nevertheless of great 
practical interest as it gives a possibility of obtaining wider stability 
conditions. 

I think there is no need to add to Figure 7(a) and (b) of my paper 
the case shown by Professor Jury in his Figure A , as much as it can be 
considered as a particular case of Figure 7(b) when the frequency 
limit is CUiim — n. 

Professor Li naturally is interested in investigations of absolute 
stability of pulse frequency systems. The criterion we proposed can be 
applied directly to the pulse frequency systems of the first order, but 
by the corresponding generalization given, it can be applied to the 
systems of higher order as well. 

I am grateful to Professor Li for mentioning the paper which 
concerns the question of investigation of stability of non-linear pulse 
systems. It will be most interesting to read this. 

In reply to Professor Ching-shuan Shue’s questions I would like 
to say that the criterion’s proof itself clearly shows that it holds, even 
when non-linear unit characteristics and sampler gain are arbitrarily 
varying. In other cases the criterion should be altered. 

If lim f[n] — const., then one cannot use the asymptotic stability 
concept for jc = 0, but must solve Liapunov’s problem or investigate 
process stability. 

The criterion is applicable even in this case. 


180 







Synthesis of Optimum Sampled-data Systems 

L.N. VOLGIN 


Summary 

In this paper the problems connected with the synthesis of optimum 
pulse systems for the control of dynamic media by means of linear 
models are considered. These problems are stated, using the mathema¬ 
tical tool of polynomial equations, which make possible the synthesis of 
efficient pulse systems. The analytical conditions for the efficiency of 
the pulse systems, which combine the usual conditions of stability of 
linear systems with the conditions of ‘approximation’ of modelling, are 
formulated. The inadequacy of the usual criteria of stability for the 
synthesis of efficient systems is demonstrated. An outline is given of the 
possible applications of the method of polynomial equations in the 
theory of automatic control. 

Sommaire 

Ce rapport traite des problemes concemant la synthese de systemes 
optimaux a impulsions pour la commande d’eldments dynamiques 
au moyen de modeles lineaires. La solution de ces problemes est 
obtenue en faisant usage des equations polynomiales qui permettent 
la synthese des systemes a impulsions. Les conditions analytiques qui 
donnent la possibility d’apprecier l’efficacite de ces systemes en combi- 
nant les conditions usuelles de stability des systemes lineaires avec les 
conditions de ^’approximation » de la simulation sont mises en evi¬ 
dence; on demontre l’insuffisance des criteres de stability usuels pour 
la synthese de tels systemes. La possibility de l’application de la rne- 
thode des equations polynomiales dans la theorie de reglage auto- 
matique, est esquissee. 

Zusammenfassung 

In diesem Beitrag wird die Synthese von optimalen Abtastsystemen fur 
die Regelung dynamischer Strecken mittels linearer Modelle behandelt. 
Diese Probleme werden mathematisch durch Polynomgleichungen 
formuliert, wodurch dcr Entwurf leistungsfahiger Abtastsysteme mog- 
lich wird. Die analytischen Bedingungen fur die Leistungsfahigkeit von 
Abtastsystemen, welche die ublichen Stabilitatsbedingungen linearer 
Systeme ebenso umfassen wie die Bedingungen fur „lSaherung“ durch 
Modell, werden ausgearbeitet. Es wird nachgewiesen, daB die ublichen 
Stabilitatskriterien fur den Entwurf leistungsfahiger Systeme un- 
geniigend sind. Die regelungstheoretischen Anwendungsmoglichkeiten 
fur die Methode der polynomen Gleichungen werden gezeigt. 


Introduction 

The transition to the scientific design of compound automatic 
complexes and the resulting increase in the calculation difficulties 
demand the finding of new ways for the formalization of solu¬ 
tions and simplification of calculation. The creation of the 
methods of linear, non-linear and dynamic programming 
should be regarded as considerable achievements in this field. 
The method of polynomial equations used below, the efficiency 
of which was demonstrated on a number of problems of auto¬ 
matic control, may be added to this number of methods. The 
polynomial equations consist of a variety of Diophant equations, 
the specific methods of solution of which are easily programmed 
for digital computers. The development of the operator method 


of analysis for the linear pulse systems, of the discrete Laplace 
transformation, or ^-transformation (see Tsypkin 1 , Gurevich 2 , 
Zadeh and Ragazzini 3 , and others), and the emergence of a 
large number of different methods for the synthesis of the 
optimum linear pulse systems (the works of Tsypkin 4 " 6 , Bergen 
and Ragazzini 7 , Chang 8 " 10 , Jury 11 , Bertram 12 , Potapov 13 , 
Krasovskii 14 , Perov 15 , and others) were the reason for the 
creation of the method of polynomial equations. 

At present the theory of optimum pulse systems for the 
control of linear plants lies at the foundation of design of self- 
optimizing systems, which contain digital computers. In these 
systems the automatic linearization of equations for the plant 
during the operation of the system is achieved on the basis of 
principles described in the works of Kalman 16 , Bigelow and 
Ruge 17 , and others. Thus, the theory of optimum linear pulse 
systems develops into the theory of an extensive class of self- 
optimizing pulse systems of control, adaptable to the changing 
characteristics of the controlled plant and to the parameters 
of external signals. 

The basic difficulties which arise in the design of the systems 
containing an optimizing model for the plant are associated 
with the violation of the conditions of ‘approximation’ of 
simulation, which require a continuous relationship between the 
quality of control and the change in the parameters of the plant 
being simulated. Some of the above-mentioned authors touch 
upon the questions of control for the plants with negative 
dynamic properties, during the compensation of which the 
violation of ‘approximation’ is possible. The criteria of approxi¬ 
mation found under these conditions, which do not allow the 
contraction of the zero and poles of the transfer function 
of the plant for the individual structures of automatic sys¬ 
tems, served as the starting point for the search of ana¬ 
lytical conditions of approximation, suitable for any struc¬ 
tures. The investigation of the conditions of approximation for 
automatic systems showed that they are closely connected with 
the conditions of stability, and that the distinctive feature of 
these conditions is based on the distinctive ideas about the 
‘’coordinate’ and ‘parameter’. The conditions found below, 
which combine the conditions of stability and approximation, 
are called the efficiency conditions , since the term ‘efficiency’ 
literally reflects the essence of the considered phenomenon. 
From the analytical conditions of efficiency for different struc¬ 
tures of automatic systems emerge different criteria for efficiency. 
On the basis of these criteria it is possible to conclude that the 
criteria of stability adopted at present are inadequate for the 
synthesis of efficient systems. The attempts to solve the problems 
of synthesis for automatic systems often encountered in literature, 
inaccurate on the whole or having a very limited field of applica¬ 
tion, are explained by this. The author has shown 18 " 20 that the 
polynomial equations represent a mathematical tool which is 
adequate for the problem of synthesis for efficient automatic 
pulse systems. A systematic treatment of the method of 
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polynomial equations is contained in the author’s monograph 20 . 
In the given paper a derivation of the analytical conditions of 
efficiency is given, and a brief survey made of the problems of 
automatic control, solvable by means of polynomial equations. 

Denotations and Terms Used 

1. Symbol z is used for transformations, where z is the delay 
operator for a single cycle. 

2. The systems and signals are represented by the rational 
real functions of z of the form F = A IB, where A and B are 
polynomials of z. 

3. The factorization of functions F with reference to contour 
/"{I z | = 1} gives the real functions F + and F = F + F~~, 
where the sign F + denotes the absence of zeros and poles 
of the function in the region D~{\ z | < l}, and the sign F~ 
denotes their absence in the region Z> + {| z | > l}. 

4. The separation of functions F with respect to contour F 
gives the real functions F = F+ + F ~, where the sign F+ denotes 
the absence of poles of the function in the region and 
the sign F~ denotes their absence in the region D + . 

5. The representation (transfer function) of the controlled 
plants will be made by G — F/Q, where P and Q are poly¬ 
nomials of z; the representation (programme) of the pulse 
unit will be made by W = C/D, where C and D are polynomials 
of z; the representation of the pulse system as a whole will be 
made by H, and the representations for the input and output 
are equal to X and Y respectively. 

Analytical Conditions for the Efficiency of Pulse Systems 


Let us find the analytical conditions for the efficiency of 
an automatic pulse system, with a single input and a single 
output coordinate, described by the following difference 
equation: 

& (x„ X; X; y„ j>,._ y t = 0 (1) 


where YF is the continuous function differentiable with respect 
to all arguments, i is the discrete time, and n and m are the 
corresponding numbers of stored values x at the input and y at 
the output. At the foundation of calculation of the system lies 
the linear model, obtainable by means of linearization of 
the equation of the system in the vicinity of the ‘operating 
point’: 


n 


z 

r = C 


f d^\ 

Wi-J 


Jo 


<> x i-k + z 

k = 0 



fyi-k 


=0 


( 2 ) 


The numbers 

** = 




(3) 


which do not depend on index / over the interval of time under 
consideration, represent the equivalent parameters of the linear 
model. 

Using z-transformation of number sequences 20 , the equation 
for the linear model (2) may be written in the form: 


Y = HX 


(4) 


where iTis the representation of the model, which is the rational 
function 


h =T’ a= I Z 

D k -0 lc = 0 


(5) 


By considering the mathematical model of an actual physical 
system, one is deliberately making a differentiation between the 
‘coordinates’ of the system, the changes in which are reflected 
by the given model, and its ‘parameters’ which are determined 
as fixed numbers which, in the given model, form the basis for 
calculations. However, the practice of construction of auto¬ 
matic systems shows that the uncontrollable discrepancies 
between the calculated and the actual parameters may be the 
cause of profound disparity in the calculated and the actual 
behaviour of the system. The failure to take this fact into 
account will sometimes lead to the construction of inefficient 
systems. The majority of automatic systems (the systems of 
stabilization and programme control, the computer and the 
reproduction systems, the systems for transmission and process¬ 
ing of data) require a continuous relationship for the behaviour 
and the small changes in external conditions, which are expressed 
in the change of input coordinates and parameters of the system. 
The conditions for which a continuous relationship between 
the coordinates of the system is observed are the conditions of 
stability. The conditions for which a continuous relationship 
between the behaviour of the system and the deviations of its 
parameters from the calculated values, which are assumed to be 
constant in a given model, is observed, are the conditions of 
approximation of simulation. The general condition of efficiency 
for an automatic system, constructed on the basis of a definite 
calculated model, which unites the conditions mentioned, may 
be formulated as follows. With small variations in the input 
coordinates and parameters of the system the variations in the 
output coordinates should be small. 


The representation of a real system, the parameters of which 
change in relation to time and coordinates, but sometimes also 
in an unexpected form, differs from the representation of its 
model by the variations dH, d 2 H, 6 Z H, ..., which must satisfy 
the general condition for the efficiency of the system. 

By varying the relation (4), the corresponding variations for 
the output of the system are obtained: 

8Y = H-8X + 8H-X 

5 2 Y=H-5 2 X+25H-SX + 8 2 H-X (6) 

The conditions under which the variations in the output coordi¬ 
nate remain small have the form: 

(<550-= 0; (<5 2 Y)_ =0; (<5 3 Y)_ =0,... (7) 

By separating the right sides of expressions (6) the analytical 
conditions for the efficiency of the pulse system are obtained: 

H_= 0; (<5H)„=0; (<5 2 H)„ =0,... (8) 

in which case the first of these conditions is the usual condition 
of stability, whereas the last are the conditions of ‘approxima¬ 
tion’ of simulation. The necessity for taking into account the 
large variations is caused by the fact that as regards the para¬ 
meters of the system its representation is a non-linear function. 
It is possible to construct an example where the violation of the 
efficiency is caused as much as is desired by a high variation 20 . 
However, in practice, mostly violations of the first two con¬ 
ditions of efficiency are encountered. 
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Criteria for the Efficiency of the Basic Structures of Automatic 
Pulse Systems 

The method of combining the controlled plants and the 
computing units is called the structural system of control. The 
simplest pulse systems of automatic control contain a single 
computing unit with representation (programme) W and a 
single controlled plant with representation G. To each 
structure of the system of control corresponds a definite func¬ 
tion H , which depends rationally on W and G : 


by proceeding from the initial function H, which is chosen 
without taking into account the conditions of efficiency. 

This assumption will be proved. By carrying out the fac¬ 
torization of the representation of the plant it is obtained that: 

G = G + G~ (14) 

Functions G + and G ~, equal to: 

G + =P + IQ + ; G~ =P~ IQ~ (15) 


H = H(W,G) 


( 9 ) 


which is called the representation of the system. For each 
structure of control there is a definite class of permissible 
functions H , which may be realized in the system by the choice 
of different control programmes W, remaining at the same 
time within the limits of conditions of efficiency. The structures, 
which permit the realization of arbitrary functions H are called 
ideal structures. The structures which do not have even a 
single permissible function are called inefficient structures. 
From the point of view of the condition of stability only 
the stable functions of type H+ are permissible functions. 
However, if it is necessary to realize an unstable function, then 
the condition of stability may be discarded by limiting oneself 
to the fulfilment of the conditions of approximation. 

By taking into account the variations in the representation 
of the controlled plant, simulated by function G , the condi¬ 
tions of efficiency (8) applied to system (9) may be written in 
the form: 


H-=0;[^-5G)=0; 




( 10 ) 


The functions H, bH/bG, b 2 HfdG 2 ,..., derived by differentiation 
of (9), depend on W and G. In synthesis of systems for the 
automatic control of the programme of the computing unit, 
W is chosen in relation to the representation of plant G: 


W=W{G) 


(ID 


The verification of the synthesized systems for efficiency is made 
by the substitution of this relationship in the expression (10) 
after carrying out the operations of differentiation in them. 

In a general case the pulse systems of automatic control 
contain several controlled plants and computing units, which are 
connected up into a single structure. These systems may have 
several inputs and outputs. The verification of the conditions for 
efficiency should be carried out in this case by the variation of 
all the output coordinates for the variation in the representations 
of all the controlled plant. 

The compensation for the negative dynamic properties of 
the controlled plant, by means of the computing unit having 
the same negative dynamic properties, is the cause of violation of 
the conditions of efficiency of pulse systems of automatic 
control. Namely, such a compensation takes place, for example, 
during the trivial recalculation of the programme for the com¬ 
puting unit W for a simple closed system, the representation of 
which is: 


by the formula: 


H-- 


W = 


WG 

1 + WG 
1 H 


G 1 -H 


( 12 ) 

(13) 


are the positive and the negative portions of the representation 
of the plant. 

The positive plant, which has a representation (7% is charac¬ 
terized by the following dynamic properties: stability, instantane¬ 
ousness of reaction, and smoothness of the transition process. 
The negative plant, which has a representation G~, displays 
negative dynamic properties: instability, retardation of reaction, 
and sudden ejections in the transition process. 

By modifying formula (12) one obtains: 


8H= 


W 


(i + woy 


• 8G ; S 2 H= — 


2 W 2 


(1+ WG)' 


■’S 2 G ;... (16) 


First of all, conditions will be found under which the closed 
system is ideal, i.e. capable of reproducing the arbitrary func¬ 
tion H. The corresponding programme for the computing unit 
is chosen in accordance with formula (13). By substituting this 
formula in (16) one obtains: 

SH = H(l-H)^;5 2 H=-2H 2 (l-H)^f-;... 


or 


5H = H(l-H ) 




8 2 H=-2H 2 (l-H) 



25P-5Q 28 2 Q\ 
P 2 + PQ ) 


;... (17) 


The conditions of efficiency (8) require that P~ = Q~ = 1. 
Thus, the closed system is ideal only in that case when the 
plant is positive. In the case of the plant with negative 
dynamic properties the function H is not realizable because of 
the violation of the conditions of approximation. 

It will be shown that the closed automatic system is efficient 
for any controlled plant, under which conditions the class of 
permissible functions of this system is equal to 

H = P~6F + (IB) 

where F+ is the arbitrary stable rational function of the form: 

F + =A/B + " (19) 


and 0 is the polynomial which satisfies the polynomial equation 
in respect of the unknown polynomials 6 and FI : 

AP-6+Q'Tl=B + (20) 


The corresponding programme of control has the form: 


W = 


AQ + 0 

p+n 


( 21 ) 
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It will be verified whether the conditions of efficiency are 
fulfilled. By substituting (21) in (16) and by taking into account 
(20) one obtains: 

, H - A6U . QSP-PSQ . 

( B + f P + Q + ’ 

2 2A 2 6 2 n Q 2 d 2 P — 2Q5PSQ + 2Pd 2 Q 

(By ' (pyo + ; 

The conditions of efficiency are fulfilled for any values of G. 
In the case of a stable controlled plant the polynomial 6 , 
as follows from the polynomial eqn (20), becomes arbitrary, 
and the class of permissible functions is extended to 

H = P~F + (22) 


by means of polynomial equations. The finding of the optimum 
programme of control is, as a rule, reduced to the solution of 
a system of polynomial equations. The computation methods for 
the solution of a system of polynomial equations applicable to 
the use of digital computers have also been developed and their 
advantage over the ordinary methods in the synthesis of con¬ 
trolled programmes for plants of a high order with complex 
correlational relationships was proved. By means of the poly¬ 
nomial equations, a number of new problems of automatic 
control, in particular for the unstable controlled plants, was 
solved. The basic problems for the synthesis of pulse systems 
and their solutions, obtained by the method of polynomial 
equations, omitting the proofs because of the lack of space, 
are now enumerated. 

The problem of synthesis of the pulse system with the mini¬ 
mum transient period for a given input action: 


Thus, one proves the criterion for the efficiency of a closed sys¬ 
tem, which requires in addition to the fulfilment of the usual 
criterion of stability, that the programme of the computing unit 
does not shorten polynomials P~ and Q~*. 

Using the analytical conditions of efficiency, it is possible 
to derive the criterion of efficiency for any structures of auto¬ 
matic systems. By means of these conditions it is easy to prove, 
for example, the following well-known propositions: 

(1) The systems on the limit of stability are inefficient. 

(2) The open systems of control are efficient only for the 
stable plants. 


X = R/S (23) 

where R and S are the polynomials of z, is reduced to the solution 
of the following polynomial equation : 


p-e + SQ~U = R (24) 

in respect of unknown polynomials 0 and 11. The corresponding 
controlling programme is equal to: 


W = 


Q + 9 
P + SYl 


(25) 


(3) The ideal structures of control for plants having nega¬ 
tive dynamic properties do not exist. 

(4) The parallel system of control is ideal for stable plants; 
the sequential (cascade) system of control is ideal only for 
positive plants. 

In view of the non-existence of ideal structures of control 
for the arbitrary plants, the criterion of efficiency of the auto¬ 
matic system is more rigid than the criterion of stability. Only 
for positive plants are the general criteria of stability of the 
linear systems adequate. 

In order not to violate the conditions of efficiency, the 
optimum function H of the system should be sought for in the 
class of permissible functions. The wider the class of per¬ 
missible functions for a given structure, the higher the quality 
of the optimum system, remaining conditions being equal. 
Therefore, in the synthesis of a system of control for a given 
plant, a structure of control having as wide a class of per¬ 
missible functions as possible, a structure close to an ideal one, 
should be chosen. 


The representation of the transient process has the form: 

E=Q~ n (26) 

The minimum duration of the transient process, which ensures 
the fulfilment of the conditions of efficiency, from the number of 
cycles, is equal to the sum of powers of polynomials P~ and Q~. 
With the limitation for the module of the controlling action: 

N<r (* = 0,1,2,...) (27) 

the corresponding problem is reduced to the finding of a non¬ 
minimum solution of the polynomial equation, which is found 
by special computing methods. The modification of the poly¬ 
nomial equation (24) leads to the derivation of a system which 
has no hidden oscillations. 

The problem of synthesis based on the criterion of the 
minimum of the total quadratic error: 

/= I «?=—.(j) E(z)E(z~ 1 )- (28) 

i =o Z7l J J r z 


The Use of Polynomial Equations in the Synthesis of Optimum 
Pulse Systems 

It has been established that the classes of permissible func¬ 
tions for pulse systems are expressed in terms of polynomial 
equations. In the author’s work 18-20 , it was shown that the 
synthesis of optimum pulse systems of control for linear plants 
based on a number of basic criteria may be made entirely 

* Applicable to the stable plants the criterion of non-contraction 
was, for the first time, introduced in the work of Bergen and 
Ragazzini 7 . 


is reduced to the solution of the system consisting of two 
polynomial equations: 

P~6 + Q~Tl=I + p-Q-\* 

p~o+u + 4>=i + p-q-$ { } 

in respect of the unknown polynomials 0, FI and (j). The 
polynomials I and U are the numerator and denominator of 

* Polynomial with the reversed order for the sequence of coeffi¬ 
cients is denoted by symbol A. 
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function X (z) X (z" 1 ). The corresponding controlling programme 
is equal to + 

(30) 

p + n 

The calculation of the quadratic error may also be made by 
means of the polynomial equation 20 . 

The problems of synthesis of the optimum pulse systems of 
automatic control and of processing of data for the random 
input signals, by taking into account the universal nature and 
the prevalence of quadratic dispersion criteria, represent the most 
favourable field for the application of polynomial equations. 
The general problem of synthesis of a pulse system, optimum 
according to the criterion of dispersion of the error for finite 
time of transition into the unshifted state, is reduced to the solu¬ 
tion of a system consisting of three polynomial equations, one 
of which secures the efficiency of the synthesized system, the 
second, the finiteness of the settling time, and the third, the 
minimization of dispersion of the error. The solution of this 
general problem determines the solutions of the numerous 
particular problems of extrapolation, filtration, differentiation 
and integration of random processes by means of pulse 
computing units. The optimization of the pulse systems, by 
arbitrary criteria of quality, is reduced to the combination of 
the method of polynomial equations and the general methods of 
mathematical programming. By means of the theory of poly¬ 
nomial equations it is possible to synthetize the most economic 
programmes for the processing of data by the method of least 
squares. The obtained results show that the polynomial equa¬ 
tions represent a suitable mathematical tool for the programming 
of many procedures of computer mathematics and of mathemat¬ 
ical statistics, which are widely used in the self-optimizing 
systems of automatic control. 

Conclusions 

The conditions of efficiency, formulated in this paper, limit 
the possibility of change in the dynamic properties of controlled 
plants by means of pulse computing units. Under these con¬ 
ditions the worst properties of the plant—instability, retarda¬ 
tion, fluctuation—are shown to be the most difficult to over¬ 
come. The limits of the accuracy of control for dynamic plants by 
means of the pulse computing units whilst being wider than 
for the units of the continuous type, are, however, not limitless. 
Physically, this means that the inertia of the plants cannot be 
completely overcome. The problem of the theory of automatic 
control lies in the further clarification of the limits of possible 
accuracy of control, and the achievement of these possibilities 


through the design of the most perfect controlling machines. It is 
hoped that the future development of polynomial equations 
will prove to be one of the important aids in the solution ol 
this problem. 
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Integral Pulse Frequency Modulated Control Systems* 

C.C. LI and R.W. JONES 


Summary 

The paper describes a new class of discrete data control systems in 
which a portion of the system makes use of pulse frequency modula¬ 
tion. This concept arose as an abstraction from the study of neural 
communication links in physiological control systems. An integral 
pulse frequency modulator is defined as one which emits a standard 
pulse whenever the integral of the input variable reaches a threshold 
value. The frequency of the output pulse train varies in a linear manner 
with the input magnitude. Modulators producing pulse trains of one 
or two signs are introduced and approximate frequency characteristics 
are obtained; they have discontinuous and discrete features, and 
depend upon signal amplitude, frequency, and initial phase angle. 
Feedback control systems making use of single-signed modulators in 
two parallel feedforward paths are described, and some of the unique 
stability considerations are studied. Continuous systems that ap¬ 
proximate these stable discrete-data systems are constructed, and 
their transient properties considered. 

Sommaire 

Le rapport decrit une nouvelle classe de systemes de commande a 
donnees discretes oil une partie du systeme utilise la modulation de 
frequence d’impulsions. Cette idee est une extrapolation de l’etude 
des liaisons de communications nerveuses des systemes de commande 
physiologiques. Un modulateur integral de frequence d’impulsions 
(IPFM) est, par definition, tel qu’il emet une impulsion standard 
lorsque Pintegrale de la variable d’entr6e atteint un seuil. La frequence 
du train d’impulsions de sortie varie lineairement avec l’ampleur du 
signal d’entree. Des modulateurs produisant des trains d’impulsions a 
un ou deux signes sont presentes et les caracteristiques approximatives 
de frequence sont obtenues. Elies ont caractere discontinu et dis- 
cret et dependent de l’amplitude, de la frequence et de Tangle de 
phase initial du signal d’entree. Des systemes de commande a reaction 
utilisant des modulateurs a un seul signe dans deux branches paralleles 
a chaine ouverte sont decrits et certaines des considerations uniques 
de stability sont etudiees. Des systemes continus approximant ces 
systemes stables de commande a donnees discretes sont construits et 
leurs proprietes transitoires sont discutees. 

Zusammenfassung 

Der Aufsatz beschreibt eine neue Klasse von diskontinuierlich arbei- 
tenden Regelungssystemen, in denen pulsfrequenzmodulierte Signale 
auftreten. Diese Darstellung ergab sich aus den Untersuchungen der 
Informationsiibertragung in Nervenbahnen in physiologischen Rege¬ 
lungssystemen. 

Ein Integral-Pulsfrequenz-Modulator wird definiert, der immer 
dann einen Impuls aussendet, wenn das Integral der EingangsgroBe 
einen Schwellwert erreicht hat. Die Frequenz der Ausgangsimpuls- 
folge ist linear zur EingangsgroBe. Modulatoren, die Impulsfolgen mit 
einem oder zwei Vorzeichen erzeugen, werden eingefiihrt. Die Ab- 
leitung der angenaherten Frequenzcharakteristiken zeigt, daB sie die 
Merkmale der diskontinuierlichen und diskreten Systeme haben und 
von der EingangsgroBe, der Frequenz und vom Anfangsphasenwinkel 
abhangen. Regelkreise, die in zwei parallelen Vorwartszweigen Modu¬ 
latoren mit einer Polaritat besitzen, werden beschrieben und einige 
Stabilitatsbetrachtungen durchgefuhrt. Kontinuierliche Systeme, die 
als Approximation dieser stabilen diskontinuierlichen Systeme dienen, 
werden abgeleitet und ihre Ubergangseigenschaften diskutiert. 


Introduction 

It has been well observed through experiments that when a 
stimulation is exerted upon a sensory organ, some form of 
energy absorption and transduction will occur, thus a neural 
signal will be generated and transmitted along the nerve fibres 
linking it to some reflex centre where a decision is made and a 
control effort is issued to respond to the stimulation 1 . The 
neural signal in a single nerve fibre is a train of pulses of ap¬ 
proximately constant magnitude and uniform shape, but its 
instantaneous frequency varies with the strength of the stimula¬ 
tion. Pulse frequency modulation is thus believed to exist in the 
neural communication networks of physiological control sys¬ 
tems 2 . In the past, only a little work was done on pulse frequency 
modulation 3 . Integral pulse frequency modulation has been 
proposed by Li and Meyer 2 * 4 > 5 . It is derived from an idealized 
model of a relaxation oscillator which is assumed to take place 
in the transduction process 2 . 

Physiological control systems contain communication chan¬ 
nels that consist of hundreds and thousands of parallel nerve 
fibres, in each of which the signal is carried by pulse frequency 
modulation. These parallel channels serve to enlarge the dynamic 
range of the transmitted signal and improve the reliability. The 
control systems described in this paper contain only single 
‘fibres’ and thus represent a drastic abstraction of the physio¬ 
logical case. 

Integral Pulse Frequency Modulation 

In integral pulse frequency modulation, the modulator emits 
a pulse of fixed magnitude and width whenever the time integral 
of its activating signal reaches a threshold magnitude. The 
integrator is reset after each pulse so that the successive inte¬ 
gration always starts from zero. An arbitrary input signal is 
thus converted into a pulse train whose repetition frequency 
varies directly with the magnitude of the activating signal. A 
simple linear and time-invariant low-pass filter is used as the 
demodulator whose output is a summation of pulse responses. 

Double-signed Integral Pulse Frequency Modulator 

Let e t (0 be the activating signal to the integral pulse frequency 
modulator (IPFM), and e f (t) be its output. The output pulses 
are assumed to be of rectangular shape. The modulator output 
is defined by 

00 

e f (t) = E f y sgn [a ( 4 )] [u(t-t k )-u(t-t k - t)] (1) 

k=l 

or, if the pulse width t is very small, each pulse can be ap¬ 
proximated by an equivalent impulse 

00 

e f (f )=M y sgn [a( 4 )] 5(t-t lc ) ( 2 ) 

k = 1 

* This study was supported in part by the grant B-2165 from the 
National Institutes of Health, U.S.A. 
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The time instant of the k th pulse or impulse, 4 ., is determined 
from 


l«(4)l = 


ei(t)dt 


= A (k = l,2, 3,...) 


(3) 


Let the input signal be a constant e(t) — E, e^t) = E + E b — E { 
f P = = j+./o provided 0<E + E b <~ (10) 


where u(t) = unit step function defined for t > 0, E f = pulse 
magnitude, r — pulse width, M = E f r = strength of the pulse 
or the impulse, <5 (7) = unit impulse at t = 0 , and 

sgn [a (4)] = +1 if a (4) = +A 

= -l if a(4)=-A 


Let T k = //. — = time interval between the (/:— 1) th and 

the /cth pulses, r < min (7^, 7" 2 , ...)• The instantaneous 

pulse frequency is defined as 

f pk = ^ (4) 

For a constant activating signal e^t) — E it 


l«(4)l = 


I 


tk- 1 


; |£ i l(4-4-i)=|£ i IT=/l 


f p =fpk= 


Ijm 

T, A 


, provided/<Lor |£ ; |<d (5) 


Therefore, pulse frequency for a constant activating signal is a 
linear function of the magnitude of the activating signal with 
1 /A as the proportionality constant, as shown in Figure 1(a). 

Since pulses of both positive and negative signs can be 
generated depending upon the sign of the time integral of the 
activating signal, this type of modulator is called double-signed 
integral pulse frequency modulator (D-S IPFM). 


Single-signed Integral Pulse Frequency Modulator 

It is well known that the pulse signals in neural communi¬ 
cation networks have only one sign. The single-signed integral 
pulse frequency modulator (S-S IPFM) is defined as one which 
can generate pulses of only one sign. This can be obtained by 
cascading a D-S IPFM to a linear rectifier as shown in Figure 1(b). 
Assume that the rectifier has a positive bias -\-E h , a positive 
unity gain and a negative cut-off value —E b . A linear rectifier 
with a negative unity gain and a positive cut-off value +j E b 
may also be used; this is introduced later when the reciprocally 
innervated parallel-path is dealt with. Let e(t) be the input 
signal to the S-S IPFM, the activating signal is 

e i (t) = e(t) + E b >0 for e(t)>-E b 1 
= 0 for e(t)<—E b j 

Hence, the output e f (t) of the S-S IPFM is a pulse train of 
positive polarity only, 

co 

e,(t)=E / £ l [«(*-4)-w(f-4-0] U) 

/c — 1 

or, by the impulse approximation, 

e f (t) = M t S(t— 4) (8) 

k=l 

where the pulse instant t k9 is determined from 


where f Q is a fixed average pulse frequency due to E b 

/o = | (11) 

The variation of the input signal e(t) about the bias level E b is 
characterized by the variation of the instantaneous pulse fre¬ 
quency about/ 0 . [This principle has been applied successfully, 
through an obviously independent work, in an analogue tape 
recorder (series 100 and 200, Mnemotron, Inc., U.S.A.)]. 

Although the pulse frequency is a linear function of the 
activating signal magnitude, the integral pulse frequency modu¬ 
lator is a non-linear and discrete device because the output 
signal and the input signal do not satisfy the additive or homo¬ 
geneous law. The precise analysis of the integral pulse frequency 
modulation is extremely difficult. 

Consider a first order system a/(s a) as a simple demodula¬ 
tor. Under a constant activating signal E t to the modulator, it 
can easily be shown that the d.c. component of the output is 
equal to the activating signal E i( multiplied by a gain M/A, and 
the fundamental ripple of angular frequency co r = 2n/\Ei\/A is 
below 3 per cent of the d.c. level when 10 aA < \E t \ < A/r. 
Under this static condition, the IPFM may be approximately 
equivalent to a continuous element of the gain M/A for 
10 • a < f p < 1 /r, where 1 fa may also be considered as the 
largest time constant in the demodulator. For S-S IPFM under 
arbitrary input e(t)> —E hi the output of its demodulator con¬ 
tains a d.c. component corresponding to (M/A) E b , the trans¬ 
mitted signal will be received by subtracting this d.c. level from 
the demodulator output. 

Among the advantages of using the integral pulse frequency 
modulator are its simple mechanisms for modulation and de¬ 
modulation, without the need of any standard clock frequency, 
and its good immunity to channel noise 5 . 


Approximate Frequency Response of Single-signed Integral Pulse 
Frequency Modulator 

Let e(t) — E s sin (cot + /?), where E s is the input signal 
amplitude, co is the angular frequency, and P is the initial phase 
angle. Several pulse distributions of e f (t) are illustrated in 
Figures 1 (c)-(g). They may be periodic, subharmonic, or 
aperiodic, depending upon whether [jj 2 q /<0 ^(0 dt]/A is equal 
to an integer q , a rational number Q/P , or an irrational number. 
The output pulse train of the S-S IPFM under sinusoidal ex¬ 
citation is periodic at the signal frequency only for a set of 
discrete frequencies such that [$ 2 o /c ° e z *(f)d t]/A — q, where q 
represents the number of pulses per signal period, 


co 

To 


2n 

q 




tk 


a(4) = J ei(t)dt=A 

tk- 1 


^ The periodic pulse distribution does not undergo any transient 
process, it reaches an equilibrium configuration right after the 
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(c) Periodic pulse distribution, E s >E b 



(d) Periodic pulse distribution, E s < E b (e) Aperiodic pulse distribution, E s < E b 



(f) Subharmonic pulse distribution, E s < E b , (P = 3, Q = 7) (g) E s < E b , a> = Zir f 0 , 


q = l, insensitive 

Figure 1. Integral pulse frequency modulators and pulse distributions of the S-S IPFM under a sinusoidal excitation e(t) = E s sin (ojt -|- ft) 


application of the input sinusoid. There is a critical value of 
ft, ft c , beyond which the periodic distribution will be equivalent 
to that for a certain ft eQ where 0 < ft ea < ft c . ft c is derived from 
o o 

j «;(i)dt=J [E b + E s sm(a>t + p c )]dt = A, 

-Pc -Pc 

co co 


A «Ec A CO E n 


There are infinitely many equilibrium configurations for ft within 
the range of 0 to ft c . After any sudden disturbance to the input, 
the pulse instants within a signal period will settle immediately 
at their new relative locations corresponding to a different value 
of ft. In this sense, each equilibrium periodic distribution is 
neutrally stable. 

The. effect of phase angle ft of the input sinusoid e(t) = 
E s sin(cot+ft) on the configuration of periodic pulse distribu¬ 
tion may be exhibited by the time interval from a zero crossing 
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time t = 0 to a reference time t 0 — (/?/co) when it is assumed 
that e(t) — E s sin co t, as shown in Figure 1. For convenience, 
consider 

e(t)=E s smojt , 0<cot Q — fi < fi c (14) 

where co satisfies (12). Referring to Figure l(c)-(d), the pulse 
instants are determined from eqn (9), 

for 




( 21 ) 


The normalized approximate frequency response or normalized 
describing function N u (co// 0 , EJE b , ft) is 


EE co E 

•^<1,0)4- -£ L coscot k = k— + l]-^cosP,(k=l,2 . q) 

E b E b /o E t (15) 

for 

JEL 


E b 


> r i0t k —^ COS cot k = k~ + p - cos p,{k = 1,2,3,... q ,) 

JO E b 

E k . _. co . _ i 1 

i -F/Z — c °s 1 + /z) - 7 :—F 7T — 2 sm r- 

Jo 1+/% 


— 2 


f 1 -i 

h b/ 


+ /?-/cos/?,(/* = !,2,...^-^) 
E b 


(16) 

where q x is the number of pulses between t = t 0 + t = fi\co + t 


and 


r=r c0 + T =- 


co 


7i +sin" 


1 

: s /£ 6 _ 


+ T 


and is determined from q, — the maximum positive integer 
equal to or less than 

\2 

-1 


1 


ffl//c 


• n 


■ -i 1 

+ sin •=-=- + 


EJE b 


L \ E b. 


i- p 

■J 


cos/? 


(17) 


For very small r, the modulator output is represented by a 
periodic impulse train at the signal frequency, 

e f (t) = M £ 8(t — t k ) for 0<—<t<—<^L7Ji (18) 


k-l 

Its Fourier series is 


co 


e f (t) = c 0 +J] c„ sin (neat 4- y„) 

n= 1 

where the coefficients are evaluated in the following way: 

(* 2 n + /f 
OJ 
t'O : 


(19) 


OJ 
2 71 


CO 


OJ _ _ 
c„=—M 

71 


/’ e f (t)dt=~qM 

— 171 

ci \2 / q 

Ycos/la)^ ) + E sin ncot k 

L\k = 1 / \fc« 1 

Cl 

E cos ncot k 


(n = 1,2,3,...) 

7»=tan _1 *-^- (n = l, 2,3,...) 

(20) 

When the fundamental component is of paramount importance 
while the higher harmonics are negligible, as may be the case 
when the low-pass demodulator is used, the approximate 
frequency response, N (co// 0 , EjE b , /?), or the describing func¬ 
tion, of the S-S IPFM is defined as the complex gain of the 
modulator with regard to the fundamental component of e f (t). 


W = Modulus ofiV d ^|^) = ^| 


1 OJ 1 


u f 0 EJE b LV/c= 


E coscot,.] +( E sin cot. 


|/V„ = Argument of N u ( + /? 

\Jo E b 


= y x = tan 


E COS cot,. 

-1 k _~_1_ 

£ sinco4 

fc= 1 


(23) 


For communication applications, it is required that E s /E b < 1. 
The normalized approximate frequency response can be com¬ 
puted at discrete frequencies for EJE b < 1 and /? = 0°. When 
q becomes increasingly larger, the difference between two con¬ 
secutive neighbouring frequencies in the discrete set is smaller; 
in this sense, the approximate frequency response of the S-S 
IPFM may be thought of as being a function of a continuous 
variable co. The effect of /? is to make N u non-unique, this 
effect is reduced as /? c is reduced when q increases. N u may 
exhibit either positive or negative phase shift. It oscillates about 
unity magnitude and null phase shift, and tends toward 1 /0 J 
as q increases or co/f 0 decreases. For example, for q > 25, 
co/f 0 < 0-08 n, N u can be approximated by 1 /0° with an error 
bounded by 1 per cent in magnitude and about 1.5° in phase 
shift. Under this operating condition, the S-S IPFM, when 
followed by a low-pass demodulator, may be approximately 
equivalent to a continuous element of gain Ml A. It is evident 
that if the parameter / 0 is made larger it will improve the com¬ 
munication fidelity. 

When q — 1, a uniform pulse train is obtained. In particular, 
if EJE b < 1, the pulses are emitted at those instants where 
normal pulses would be even if e(/) == 0; the S-S IPFM becomes 
insensitive. 


Parallel-Path Single-signed Integral Pulse Frequency 
Modulated Control Systems 

A new class of discrete-data feedback control systems is 
formed when integral pulse frequency modulators are intro¬ 
duced for system error modulation 5 . The parallel-path single- 
signed integral pulse frequency modulated control systems 
(P-P S-S IPFMCS) are presented here. The block diagram is 
shown in Figure 2(a). There are two feedforward paths in 
parallel, each containing one S-S IPFM. These two modulators 
have the same values of E b , A , E f and r, but one has a linear 
rectifier with negative unity gain and positive cut-off value +iT 6 . 
The output of the first modulator, e fl (t), is to drive the system 
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G^s), and the output of the second modulator, e f2 (t), the system 
G 2 (s). G^s) and G 2 (s) serve as demodulators, their respective 
outputs e dl (t) and e d2 (t) are subtracted one from the other; 
e d (t) — e dl (t) — e d2 (t) is to drive the final controlled element 
G(s). r(J) and c(t) are the input and output of the system 
respectively. When the system error e(t ), also the input to the 
modulators, is zero, there is a constant pulse frequency / 0 in 
each path; the average forces or torques developed in both 


paths are virtually cancelled out at the subtractor if G^s) and 
G 2 (s) are identical. When e(t) is non-zero, it increases the pulse 
frequency in one path while decreasing it in the other; the 
deviations of pulse frequencies in the two paths cause the 
developed force in one path to be higher and that in the other 
lower; both are in the same direction. The net control effort 
results from the pushpull action of the two paths. This char¬ 
acteristic is termed reciprocal innervation in the physiological 



(a) PARALLEL- PATH SINGLE - SIGNED INTEGRAL PULSE FREQUENCY MODULATED 
CONTROL SYSTEM 



(b) APPROXIMATELY EQUIVALENT CONTINUOUS SYSTEM OF PARALLEL-PATH 

SINGLE-SIGNED INTEGRAL PULSE FREQUENCY MODULATED CONTROL SYSTEM 

Figure 2. Block diagrams of the parallel-path single-signed integral pulse frequency modulated control system and its approximately 

equivalent continuous system 
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context. Thus, the S-S IPFM’s can be used for error modulation 
and at the same time the object can be controlled in either 
direction. The system is insensitive to small, high-frequency 
disturbances and behaves without a dead zone for slowly varying 
signals; in effect, it is actively controlled at all times. 

Some salient phenomena have been noted when the pulse 
frequency is relatively low. This case is interesting because the 
nerve impulse in physiological systems varies approximately 
from 0 to 1,000 pulses/sec. While the actual physiological con¬ 
trol loops are complex far beyond our present knowledge, this 
discussion is confined to some simple abstract models, as shown 
in Figure 2(a). All components except S-S IPFM’s are assumed 
to be continuous and linear. G ± (s) and G 2 (s) may be (a) identical, 
or (b) may simply have the same number of poles, the same 
number of zeros, and the same static gain: 


G 1 (s) = k 1 
G 2 (s) = k 2 


S m +b 11 5 m ~ 1 

s n + a ll s n ~ 1 

s ,,, + fo 21 s m ~ 1 
s n +a 2l s' 1-1 


k, 


^2m 

a ln a 2n 


+ ... +b lm 
-b ... +a ln 

+ — +fr 2 m 

F ••• +«2 n 


(24) 


The former is called the balanced parallel-path, and the latter 
the specially unbalanced parallel-path. Their dynamic char¬ 
acteristics and an approximate method of analysis are presented 
in the following. 


Stability Analysis 

Consider e(t) = 0 , e fx (t) and e fz (t) are two uniform pulse 
trains with pulse frequency / 0 . The d.c. levels in e dl (t) and 
e d2 (t) are the same; the ripples are of the repetition frequency 
/ 0 . e d (t) and c(t) will contain a small oscillation of repetition 
frequency / 0 , unless e fl and e f2 are synchronized for a balanced 
parallel-path system. Although c(t ) will be fed back, e fl (t) and 
e f2 (t) are unaffected, providing that the feedback signal is 
bounded by E b . This can be deduced from Figure 1(g). Thus, 
an equilibrium state can be established. This is defined as the 
static equilibrium state for the P-P S-S IPFMCS. There are 
many such static equilibrium states, each is neutrally stable, 
with regard to the degree of asynchronization of the two uniform 
pulse trains. After a sudden disturbance to the system, c(t) 
may finally tend toward one of its static equilibrium states. 
This is termed the ‘almost asymptotic stability’ of the P-P S-S 
IPFMCS. It is formally defined as follows. If the system output 
is disturbed at t 0 from c(t) = 0 with c(/ 0 ) < rj < oo, there 
exists a small non-negative number a depending on rj such that 
\c(t)\ < o-; o' is bounded by a prescribed value, sufficiently 
small, which represents the maxima of the tolerable ripple in 
c(t) due to the two asynchronous pulse trains of frequency f 0 . 
This definition is in contrast with the asymptotic stability of 
the continuous system where o-> 0 only. The concept ‘almost 
asymptotic stability’ is necessary for the description of control 
systems of this class. 

Application of Normalized Describing Function N u 

The normalized describing function N u (co/f 0 , E s /E b , P) has 
been derived in eqns (22) and (23). A portion of — (l/N u ) is 


plotted in the polar form as shown in Figure 3. q contributes 
the discreteness of the consistent pair of (to// 0 , EJE b ), for each 
of which, the variation of P from 0 to p c traces out a curve for 

— 1 jN u which shows a sharp change or discontinuity at a 
certain value of /?. Through extensive numerical computations, 
all these two-parameter curves — [l/N u (o>// 0 , EJE b )] seem to 
be symmetric with respect to the real axis, and may cross over 
one another. For q= 1 , 0 < EJE b < 1 and co// 0 — 2 tz, 

— (1 /N u ) simply occupies a circular region of radius 0*5 and 
centred at the origin, circles with small radii correspond to 
small EJE b ; for q = 1, EJE b > 1 and co // 0 >2 tz 9 the mappings 
are concentric circular arcs with radii within the range of 0.5 
to n12. For q > 2, the curves are situated to the left of a bound¬ 
ary to which — [l/N u (cn/f 0 = tz, E s /E b = 0)] is asymptotic. 
When EJE b < 1, the curves are convergent toward the — 1 
point as q increases or co/f Q decreases. When EJE b > 1 and for 
a given q, the curves corresponding to larger EJE b and larger 
o )// 0 are located further to the left and may cross the negative 
real axis to the left of the — 1 point. Because r > 0, pulses may 
theoretically reach complete saturation in the positive half 
cycle with no pulse in the negative half cycle of the sinusoid 
when E s /E b is large enough; the modulator characteristics would 
approach that of a relay of one polarity with small hysteresis. 
The limiting plot of — (1 /N u ) is then a narrow region with 
negative real axis as its centre line and extending to the left of 
the — 1 point. 

The above discussion can be similarly applied to the S-S 
IPFM whose linear rectifier has a negative unity gain; its 
normalized describing function has a negative sign to that 
given by eqn ( 22 ), in addition, P is changed to ft + 180° which 
will be made equivalent to a proper P eQ = p' < p c . 

The describing functions of the modulators in the parallel- 
path are respectively 


Mi- 


and 


( co E s 

T/o’-eF 


A* u \f 0 ’E b ’ P 


For describing function analysis, R (jo) = 0, C (jo) = — E(jco ), 


C (jeo) = G ( jeo ) [E d 1 {jco) - E d2 (jeo )] 

M f (co E, 

= —GOco) GiOco) 

'•fef/)] 


+ G 2 (jeo) N u 


,/o’V 


E(jco) 


and 


^ G (jeo) G, (jeo) N u (j-,pj + G 2 (jeo) N u (j-, /?') 


= -l 


(25) 


The signal fed back to the modulators is not purely sinusoidal, 
p and ft' of the fundamental components in the two paths may 
change from cycle to cycle. For the parallel-path systems under 
study, the phasors of G 1 (yco) and G 2 (jo) are rather close for 
very large and very small co. It is proposed that let P = p' so the 
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90 ° 



270 ° 


Figure 3. Describing function analyses of parallel-path single-signed integral pulse frequency modulated control systems 


bracketed term in eqn (25) would be a maximum to cause the 
most severe stability situation. The critical equation is then 

-j G (j(Q) [Gj (;©) + G 2 (jo)] =- -M- —- (26) 

A „ / co E s n \ 


The transfer locus Ml (A) G(fto) [G^/co) + G 2 (jco)] is graded 
by co// 0 for a given f 0 of the specific modulator. Any possible 
intersection of this transfer locus with any — 1 /N u curve at a 
coincident value of co// 0 represents a possible oscillation at that 
angular frequency co and of the amplitude E s determined from 
E s /E b graded on that intersecting curve of — 1 /N u . 

It is postulated that the approximated criterion for the 
‘almost asymptotic stability’ of the balanced or specially un¬ 
balanced P-P S-S IPFMCS is that the transfer locus does not 
intersect with ~\jN u curves at any coincident frequency co// 0 
except inside the circular region of a radius sufficiently smaller 
than 0*5 where q = 1, as illustrated by the system 1 in Figure 3. 
The latter intersection indicates the small steady state ripple, 
which may exist for such a stable system, of frequency / 0 and 


amplitude approximately bounded by the value of E s determined 
from the intersecting circle; E s is required to be below a specified 
value for smooth performance of the system. 

If the transfer locus intersects, in addition, at other coincident 
frequencies co/f 0 with — 1 /N u curves for q > 2, as illustrated by 
the system 2 in Figure 3, the P-P S-S IPFMCS is considered 
unstable and will exhibit a strong self-sustained oscillation. 
There may be several intersecting points. Such an intersecting 
point, obtained only for a discrete set of (co// 0 , EJE b \ would 
theoretically represent a pure sinusoidal oscillation. But real¬ 
istically, the harmonics in the system output will be fed back 
to the modulators; not only the pulse distribution in the succeed¬ 
ing cycles will change, but the number of pulses per oscillation 
cycle will also vary. This effect is cumulative cycle after cycle. 
It may account for the mechanism of formation of almost 
sinusoidal, subharmonic, or aperiodic oscillation. The transfer 
locus of the system 2 crosses the negative real axis at about 
—1*46 with co = 1*57 rad/sec or co// 0 = 0*314; the intersections 
may occur in this neighbourhood, A self-sustained aperiodic 
oscillation is observed as shown in Figure 4(b) where co varies 
from 1-5 to 1*65 rad/sec and q varies from 26 to 20. 
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Time in seconds 

I, . ... » 1 —r—) — t t '■ I - t 1-4-4- 

e f|M (b) THE RESPONSE OF SYSTEM 2 AFTER AN INITIAL CONDITION 

[c(o)«IO, c(o)»0, e dl (o)-0, e d2 (o)*o] 


“““ 0 - 


e f2 (t) 


10- 
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Time in se conds | 

(a) A STEP RESPONSE OF SYSTEM I 


FOR SYSTEM |: G, («) • G z («) - , 

FOR SYSTEM 2*• G, (*) - G 2 («) - , 


10 


G(«) 

" SlS+5) 


S(S + 5) 
50 


A - 2 VOLT/SEC, 
Ef-20 VOLTS 


E b - 10 VOLTS, f 0 - 5 PULSES/SEC 

T■ 0.01 SEC, M - 0.20 VOLT/SEC 


Figure 4. Time responses of two parallel-path single-signed integral pulse frequency modulated control systems. Their describing function 

analyses are shown in Figure 3 


Approximately Equivalent Continuous System 

When a P-P S-S IPFMCS is ‘almost asymptotically stable’, 
an approximately equivalent continuous system can be postulated 
as shown in Figure 2 ( b ) where a simple gain of Ml A with 
saturation level E f replaces each IPFM. The transient response 
due to the well defined deterministic input may be satisfactorily 
estimated from this continuous system approximation. An 
analogue computer study for the system 1 is illustrated in 
Figure 4(a ), c(t) is shown almost the same as the output of the 
continuous system c eq (i). 
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DISCUSSION 


E. I. Jury and T. Pavlidis, University of California, Berkeley, Cali¬ 
fornia, U.S.A. 

The authors suggest a pulse frequency modulation scheme which is an 
idealized model of the neural communication. A more realistic model 
involves a more complicated form of pulse frequency modulation. One 
of the purposes of this discussion is to present such a generalized 
model. The form of this modulator is such that it emits a pulse when 
the convolution of the stimulus S(t) with the weighting function h(t) 
reaches a certain threshold, i.e. 

' S(Oh(t-Z)dl;=A (1) 

0 


It is of interest to note that the authors’ model is obtained as a 
special case from (1) by letting h(t) = 1. Furthermore, we obtain a 
system with a strength-duration curve which admits a non-zero 
threshold. This curve is well known to physiologists. 

The general modulation scheme not only represents a better 
model of a neural communication but it also has advantages when 
used in feedback control systems. This becomes evident by observing 
in Figure A, that H(s) could play the role of a compensator. Further¬ 
more, such a system could improve the filtering properties of the feed¬ 
back system more than the IPFM. 

In the stability investigation of IPFM, the authors have used the 
method of the describing function. This method while valid, leads to 
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a very complicated graphical and analytical procedure. It also fails 
to apply to the general case proposed by the discussors. Hence, a new 
stability procedure is attempted and has been successfully applied, 
which is based on the quasi-describing function. This function is 
derived by using a square-wave input to the non-linearity and obtaining 
an estimate of the ratio of the fundamental harmonic of the output 



Figure A 


of the non-linearity to the fundamental of the input. This is presented 
by the following equation: 

e(s,G))= ^f{ 0 Ty(0dt (2) 

where co = 2 n\T, T is the square-wave period and S 0 its amplitude. 
This general method has been applied to the authors’ system and the 
results are shown in Table l in comparison with the results obtained by 
using the describing function and the exact experimental model. 
Although the describing function yields slightly more accurate results, 
the labour involved is considerably more than when using the quasi¬ 
describing function. 


Table 1 


Linear plant 

CO) 


Analogue 

computer 

Describing 

function 

Quasi-describ¬ 
ing function 

5 

CO 

0*965-0-998 

0*93-0*97 

0*95-1*0 

5 (s + 0-2) 

5 

30-3*2 

2*7-2*96 

4 

10 

CO 

1*3 

1*2-1 *3 

1*0-1 *4 

(s + 0-2) (s T 0-5) 

S 

1 *5-2*0 

—* 1*95 

1 *2-2*1 

10 

CO 

1*43-1*46 

1*38-1*41 

^ 1*4 

s 2 + 0-24 s + 0*16 

5 

5 *4-5 *7 

4*7-5 

6*3 


In conclusion, we commend the authors for their initial study of the 
system which stimulated our investigation of more complicated and 
realistic models. It is hoped that future studies will shed more light on 
the problem and also lead to further study of neural nets. Such a study 
will point out the value of this idealization. 
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C. C. Li, in reply 

We thank Professor Jury and Mr. Pavlidis for their interesting dis¬ 
cussion. Through their courtesy, we have received and read the com¬ 
plete report cited in their Reference 1. It is indeed encouraging to learn 
that our work has stimulated their research interests for generalization 
and improvement. 

We also take this opportunity to mention two recent contributions 
related to our paper. 
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Professor Li’s paper is a theoretical treatment of the transfer behaviour 
of neural systems by considering the pulse frequency modulation of the 
information which occurs in such systems. In his theoretical abstrac¬ 
tion, rectangular pulses are used for modulation; the line is ready to 
transmit a new pulse immediately after the end of the previous pulse. 
The nerve pulses have a quite different shape: a triangular pulse with 
a duration of about 1 msec is followed by a much longer refractory 
period during which the line will not transmit pulses. Thus, if the 
exciting frequency increases, only every second, third, etc. pulse will 
be followed by a nerve pulse. Furthermore, it is known that the shape 
of the neural pulse with its refractory period determines substantially 
the transfer properties of the neuron. Thus it is doubtful whether the 
theoretical considerations of Professor Li are transferable to the study 
of the behaviour of neural systems in this form. Possibly, thorough 
reconsideration will have to be given to the theory with regard to the 
physiological conditions. 

C. C. Li, in reply 

The authors agree with Dr. Vossius’ important remark from the 
realistic physiological point of view. As pointed out in the outset, the 
present paper concerns only the study of certain dynamic behaviours 
of the abstract systems including the proposed pulse-frequency modu¬ 
lators. The neural pulse in reality is neither a rectangular pulse nor a 
triangular one. Probably it could be described by the output from a 
non-linear network coupled to an IPFM such that the refractory 
period might be adequately taken care of. No doubt, extensive studies 
will have to be made along this direction. (Professor Jury mentioned 
that he has obtained some progress in this respect.) 

M. Hamza, ETH Zurich , Switzerland 

In the field of sampled-data control systems pulse width and pulse- 
frequency modulation are gaining in importance. This is due to their 
wide range of applications. Unfortunately, the analysis of systems 
employing such samplers is very difficult because the latter are highly 
non-linear. Very few publications have appeared on sampled-data 
control systems having a variable sampling frequency 1 . This work is 
novel and contains very useful ideas for which the authors are to be 
highly congratulated. 

In the Institute of Automatic Control and Industrial Electronics of 
the ETH, control systems of the above type have been studied. I have 
considered pulse-width and pulse-frequer.cy modulation as a means of 
compensating control systems 2 , and the results obtained were very 
encouraging, some of which—in addition to newer ones—will be 
published soon. 

I would like to know if the authors have obtained any results on 
means of improving system response using the samplers they have 
described? Have they considered using the PPSSIPF modulator in an 
adaptive control system? Can they suggest any method for obtaining 
the time domain system response other than by analogue computer 
simulation ? 

Assuming the system could be approximated by a continuous one, 
what methods do they recommend for obtaining the system response ? 
Finally, how should one select G x (s\ C 2 U)? 
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C. C. Li, in reply 

Dr. Hamza’s discussion is greatly appreciated. In answer to the ques¬ 
tions raised, I would say the following. The work reported in this paper 
essentially concerns the stability properties and salient features per¬ 
taining to the particular class of pulse-frequency modulated control 
systems. The IPFM itself is used as a device of transmitting signals, 


not as a sampler. Studies have been made for a certain adaptive 
P-P S-S IPFMCS incorporated with a reference model and for a 
varying rate sampled-data control system where an IPFM is used to 
trigger the sampler. With regard to the time domain system response, 
no method has yet been found to obtain the exact time response 
except the estimation based upon continuous system approximation, 
whenever permissible. In the system, G x (s) and G 2 (s) represent two 
continuous components satisfying conditions stated in the paper, their 
selection has not been discussed. 

I am very glad to know that Dr. Hamza has carried out researches 
in this field, and look forward to interchanging our new findings and 
ideas. 


195 


Combination of Finite Settling Time and Minimum Integral 
of Squared Error in Digital Control Systems 

V. PETERKA 


Summary 

The design of the digital controller frequently follows the aim to 
obtain the system with the fastest response. By the practical application 
of this criterion, cases are often encountered where the transient error, 
caused by an input of typical form, has a very short duration but an 
inadmissible magnitude. This shortcoming is removed by the numerical 
method of controller design presented in the paper. 

In this method the finite settling time is combined with the mini¬ 
mum of integral squared error. In this integral either the same impor¬ 
tance is allotted to all errors during the control process, or only the 
errors occurring after the first sampling period are being considered. 
The physical meaning of the second case is the requirement of the 
computer liquidating the error, as far as possible, during one sampling 
period, and not instantaneously. The paper shows that good results 
can be obtained particularly by the second method. The method of 
calculating is arranged in such a way that both alternatives can be 
investigated simultaneously. The author succeeded in arranging the 
calculating procedure into a simple scheme. A numerical example is 
given. 


Sommaire 

Les circuits numeriques de reglage sont construits habituellement de 
maniere, a ce que la dur£e du regime transitoire soit minimale. Dans 
l’application pratique de ce critere on trouve souvent des exemples oil 
la duree du regime transitoire est courte, mais l’ecart transitoire atteint 
des valeurs inadmissibles. Cet inconvenient est supprime par la me- 
thode numerique de calcul du regulateur dtaite dans ce rapport. 

Dans la methode decrite on combine le critere de duree finie du 
regime transitoire avec la condition consistant a minimaliser l’integrale 
du carre de l’ecart. Dans cette integrate cn tient compte soit de tous 
les ecarts depuis le commencement du regime transitoire, soit seule- 
ment des hearts apres le premier intervalle d’echantillonnage. On 
montre dans ce rapport que souvent cette seconde alternative conduit 
a de meilleurs resultats. On a reussi a arranger le calcul dans un 
schema simple, permettant de calculer en meme temps les deux cas. 
La m&hode est d6montrde sur un exemple numerique. 


Zusammenfassung 

Der Einsatz von digitalen Reglern ist haufig damit begriindet, Systeme 
mit minimaler Ausregelzeit zu bekommen. Bei der praktischen 
Anwendung dieses Kriteriums kommt es oft vor, daft fur typische 
EingangsgroBen zwar die Ausregelzeit sehr kurz ist, aber die Regel- 
abweichung wahrend des Ubergangsvorganges unzulassig grofie 
Werte annimmt. Diese Arbeit enthalt ein numerisches Verfahren, das 
diesen Nachteil beseitigt. 

Die vorliegende Methode kombiniert die endliche Ausregelzeit mit 
der Forderung des minimalen Integralwertes der quadratischen 
Abweichung. Dieses Integral beriicksichtigt entweder alle Fehler 
gleichwertig wahrend des Regelvorganges Oder nur diejenigen, die 
nach der ersten Tastperiode auftreten. Die physikalische Bedeutung 
des zweiten Falles ist die Forderung, daB der Digitalregler die Regel- 
abweichung nicht sofort, sondern nach einer Tastperiode bestmoglich 
beseitigen soli. Der Aufsatz zeigt, daB sich gute Ergebnisse besonders 


durch die zweite Methode erzielen lassen. Der Aufbau des Berech- 
nungsverfahrens gestattet beide Moglichkeiten gleichzeitig zu be- 
rechnen. Ein numerisches Beispiel ist angefiihrt. 


Introduction 

The aim frequently followed in the design of digital control 
systems is to eliminate the system error, caused by an input signal 
of a typical form (step, ramp, constant acceleration), within a 
minimum time 1-3 . Cases are often encountered in such systems 
with the fastest response where the transient error has a very 
short duration, but an inadmissible magnitude. This shortcoming 
can be removed by extending the response by one, two, or more 
sampling periods, as required, and by the application of a further 
criterion suppressing the system errors 4 . This article deals with a 
simple numerical method of digital controller design where the 
criterion of finite settling time has been combined with the mini¬ 
mum integral of squared error. 


The Statement of the Problem 

Consider a control system compensated by a digital con¬ 
troller according to Figure 1. It is assumed that the transfer 
function of the plant is a rational fraction 

m 

y (I, 

I *.P- ft (p-PJ 

v = 0 v — 1 

with all its poles p v in the left-hand side semi-plane p , or with 
maximum one pole equalling zero. For simplicity, let the prob¬ 
lem be confined to an input signal having the form of a unit step 


W(z) = 


1 

1-z" 1 


and the holding device being of zero order 


i*(R) = 


l-e- r * 

P 


Cases with another type of input signal, and with a holding 
device of a higher order, can be investigated in a similar way. It 
will also be assumed that the time required for the computing 
operation can be neglected, and the system has no dead time; 
however, it is possible to show that the consideration of both 
these lags is possible without any fundamental difficulties. 
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Figure 1 


Let the pulse-transfer function of the continuously acting 
member be denoted 


Q(z)= — 0 + ^i z 1 + *«•+ n B(z) 
A 0 + A 1 z~ 1 + ...+A„z~"~ ^-( z ) 


( 2 ) 


The conditions of a finite settling time have been discussed 
in detail 1-4 and here they are stated only briefly in a form suited 
for the case. 

For attaining a zero steady-state error at the sampling in¬ 
stants, after a finite number of sampling periods and under the 
conditions stated above, it is necessary that the overall pulse- 
transfer function 


F(z) = 


P&G (z) 

1 +P(z)G(z) 


( 3 ) 


should be a polynomial in z~\ and 


F (1) = 1 (4) 

If the intersampling ripples also are to be eliminated, it is 
necessary to attain the settling of the manipulated variable y(t). 
This will happen, if the pulse-transfer function E 2 (z) j W(z) 
is also a polynomial in z~ x . The equation for this transfer func¬ 
tion can be modified by the relations 


magnitude as shown in the example that follows. Therefore let 
polynomial D(z) be of the general order of L , and state the 
problem as the determination of the coefficients D 0 , D l9 ..., 
Dl of the polynomial with the integral of the squared error 

J=j°° q(t)el(t)dt (10) 

having a minimum value. 

Now it remains to select a suitable weighting function q(t), 
so the following cases will be investigated. 

In the first the same importance is allotted to all errors 
during the control process and q(t) — 1 is selected, Figure 2(a). 
However, this selection need not be necessarily the most ad¬ 
vantageous, namely the largest share in integral (10) belongs to 
errors at the beginning of the control process that cannot be 
physically eliminated in plants with a step function response 
starting from the origin. The minimalization of integral (10) can 
produce rather large overshoots that are not always desirable. 



into the form 


E 2 {z) = 


X(z) 

G(zY 


F(z) = 


X(z) 
W(z ) 


E 2 (z)_F(z)_F(z)A(z) 
W(z) G(z) J3(z) 


( 5 ) 


It follows from eqn (5) that all conditions stated will be fulfilled, 
if the overall pulse-transfer function has the form 


F(z) = -E)B(z)D(z) (6) 

where 

D(z) = D 0 + D x z ~ 1 + ...+D l z~ l (7) 

is a selectable polynomial for which 


0(1)=IA=1 (8) 

i = 0 

From relations (3) and (6) the necessary pulse-transfer function 
of the digital computer follows 


P(z)= D(2M ( z) . 

w B()-)~D(z)B(z) 


( 9 ) 


If D(z) = 1 is selected the system will have the fastest re¬ 
sponse, nevertheless the transient error can reach an inadmissible 


For this reason it is necessary to investigate the second case 
where no errors in the first sampling period are contained in in¬ 
tegral (10), i.e. the weighting function is selected in the form of 
a unit-step function in time F, q(t) = 1 (t — T) Figure 2(b). The 
physical meaning of this condition is the requirement of the 
computer liquidating the error, as far as possible, during one 
step, and not instantaneously as demanded in the former case. 
That is to say, in this second case the requirement put forward 
is less severe, and technically easier to realize. 

The method of calculation is arranged in such a way that 
both cases can be investigated simultaneously, and thus it is 
possible to reach a decision in favour of the case that is more 
beneficial at the given concrete application. 


The Survey of Results 

Coefficients D lt Z> 2 , ... 9 Dl of the selectable polynomial (7), 
fulfilling the condition of the minimum integral (10) of the 
squared error, can be found by the solution of the system of 
linear equations 

[KJ [/),]= ~[iU (ID 

where the square matrix [K rs ] is symmetrical the elements of 
which, and also the elements of column matrix [R r0 ], are in¬ 
dependent of the selected degree L of polynomial D(z). Two dif¬ 
ferent cases are considered in the calculation of the elements of 
matrices [K rs ] and [F r0 ]: (a) the transfer function S(p) of the 
plant has no zero pole, and (b) S(p) has one zero pole. 
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Case (a) 

In the case of the transfer function S(j>) having all its poles 
different from zero, the step function response of the system is 
given by the equation 

sW = i?- 1 j^M = c 0 + £ c v e** (12) 

l P J V — 1 


A slip of paper is laid beside the column with the coefficients 
A n , A n - V ..., A 0 of the denominator of pulse-transfer function 
G (z) written on it one below the other. The product of figures 
lying beside each other (see column (a) of Table 1) supplies the 
value of 

(17) 

i — 0 


In this, and in all other equations that follow, the assump¬ 
tion is made that all the poles differ from each other. The case 
of multiple poles can be introduced by means of limits. The 
elements of matrices |X r J and [jR r0 ] are calculated by the fol¬ 
lowing procedure. First of all the following expressions are 
solved numerically 


where 


0(k)= £ p v z k , 


fe = 0 , 1,2 



5 V = i 


= i p x + p 


(13) 


(14) 


The calculation is made for k = 0, 1, 2, ..., n + L , where L 
is the selected degree of polynomial D(z). 

The procedure is continued in such a way that all elements of 
the same row of matrix [K rs ], and also of matrix [R r0 ], are cal¬ 
culated simultaneously for the weighting function q(t) = 1, and 
also for q(t) = 1 (t — T). As [K rs ] is a symmetrical matrix, it 
is sufficient to calculate the numerical values only of the elements 
lying below and on the main diagonal. 

In order to calculate the elements of the rth row, the follow¬ 
ing equations have to be solved numerically 


r T- k = 9(k) — 0(r + k), k = 0,1,2, ...,n 

r F k = c-min (k, r) + 6(k) — 0{\r — k\), k=l,2, ...,n + r ^ 


where 

c=C 2 0 T (16) 

and mi n(k,r) denotes the lower of the numbers k, r. 

The figures r /A w .. . r r n+r obtained in this way are entered into 
column (a) shown in Table 1. 


Table 1 


(b) 


r U 0 

r Uj 


Rr 


(a) 


rp 


An 

r p 

l ~ n + k 


Ao 

r u s - 

A n — i 

r p 

1 -n + lc+l 


A 1 

r u s+ 1 

•^l 

T 

i—( 

i 


An— l 

r Us+n- 1 

A 0 

r r k - r u k 


A n 

r U s+n 


r p 

j- n -j- r 


T Un+r 


R 


rs 


which is then entered in the next column into the row containing 
the coefficient A 0 . All the required values of r U k (k = 1,2,..., 
n +r) are then obtained by a gradual shifting of the paper slip 
with the coefficients A { written on it. 

The next operation represented by 

n 

£«=£ A, r u t+ . (18) 

i = 0 

is performed again by using the paper slip with coefficients A t . 
However, this time the coefficients are written in the opposite 
order as can be seen from column (b) of Table 1 where the paper 
slip is drawn in a position at which the numerical value of R rs is 
being determined. The value R r0 is already the requested element 
of column matrix [R r0 ] for the weighting function q(t) = 1. The 
elements of matrix are obtained simply as the difference 


K n = R„-R r o (19) 

The correctness of the calculations made so far can be 
checked by the relation 

/» v 

( 20 ) 


J£„+2l? r0 = rc(_X 


The elements of matrices [R n ] and [/7 r0 ] pertaining to the 
weighting function q(t) = 1 (t — T) are obtained by adding the 
same figure of A K respectively A R to all elements of matrices 
[K rs ] and [i? r(1 ] respectively. 

£..=K r ,+AK 


Vs xv rs 

JR. r0 = R r0 + AR 

The figures to be added are obtained from 

AK= -AAq 


AR — XA'q-\-kAq 


( 21 ) 


( 22 ) 


where 


i = i 




(23) 


X=c-9(0 )-C 0 iAic,z t (^ 
J y 

" „=1 Pv + Pn 

K= £ 

V= 1 Pv 


Case ( b) 

In the case where one pole of transfer function S(p) lies in 
the origin, the step function response of the system is given by 
the equation 


5(0 = ^" 1 




Cif + C 0 + Yj C v e Pvt , N = n -1 (24) 

V = 1 
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The elements of matrices [K rs ] and [R r0 ] are calculated by the 
same method, only some values are calculated according to 
changed formulae. 

Now, the numerical values of 6(k) for k — 1, 2, . N+-L 
(where N — n — 1 is the number of non-zero poles) are ob¬ 
tained from the relations 

k> 0, 0(fc)=y Pv zr i (25) 


p v = C v (l-z v ) 2 


C_! Co 


——<5, 

Tv 


z v = e PvT , 

The value of 6(0) is calculated separately from 


0 ( 0 ) = - 


-CqT—IC-xT £ — — 2 X - 

v=l Pv v=l 1 ■ 


The further procedure of calculation remains the same, ex¬ 
cept that for c we substitute everywhere 

t^CiiT 3 (28) 

and instead of coefficients A { (/ =0, 1,«) we use everywhere 
the coefficients A l (/ — 0, 1,2, N). Their relationship can be 

seen from the arrangement of the denominator of the pulse- 
transfer function G(z) 

A (z) = A 0 + A iZ 1 + ... + A n z 

= (1 —z *)(Aq-)tA}Z 1 + ... +/l^z N ) , N = n — 1 (29) 

This arrangement is made possible just because one pole of the 
transfer function S(p) equals zero. 

The last difference in comparison with case (a) lies in the 
determination of the numerical values of 2 and k which are 
used in the determination of the matrices pertaining to the 
weighting function q(t) — \ (t, — T). They are calculated from 
the formulae 


A =-L+C o C_ 1 r 2 -0(O)- £ C v (l — z v )( — ~ 

Z v= 1 \ Pv 

§ . y c *(l-z„) 

v +1 Pm + Pv 

k =Z+C 0 C_ 1 7' 2 -C_ 1 T £ ^(1-Z V ) 

z v — 1 Pv 


Example 

In order to illustrate the method of calculation described 
generally in the preceding section, the calculation of a concrete 
case is given below. The transfer function of the plant is 

C/- 6p+4-5 

MW ~Cp + 2)Cp+l)(p + 0.5) 

All poles of this transfer function are different from zero, the 
problem discussed is thus of the type of Case (a). The unit-step 
function response of the system is 


s (0 = - 1 1^~} = Co + C! e p " ' + C 2 e P2t + C 3 e M ‘ 

Pi — “2; p 2 — — 1; p 3 = -0-5; C 0 =4-5; C 1 = 2-5; C 2 = — 3; 
C 3 = — 4. 

The continuously acting member of the system has a pulse- 
transfer function 

B (z)_ 1-309 z~ 1 --0-092 z~ 2 +0-248 z~~ 3 
^ Z ~~ A (z)~ i _ 1-110 z~ 1 +0-355 z -2 —0-030 z“ 3 


Let the calculation of the coefficients of polynomial D(z) for its 
selected degree L = 1, 2, 3 be presented. By solving eqns (14) 
and (13) we obtain 



79 

10 . 
“ 3 ’ 

<$3 = 5 


p 1= -0-6875; 

Pi = 

3-5; p 3 = 16 

k 

= 0 1 

2 

3 4 5 6 

Hk) 

= 18*813 10*899 

6*347 

3-743 2-229 1-337 0-805 


Table 2 contains the calculation of the elements of the second 
row (r = 2) of matrices |X rs ] and [J^ r0 ] - 


Table 2 


k 


2 t4 

l*Vc 

*2,0 

-3 

2-406 




-2 

4-118 




-1 

7-156 




0 

12-465 

5-913 

-0-694 


1 

20-250 

8-833 

0-937 

1-630 

2 

28-035 

9-771 

2-567 

3-260 

3 

33-344 

9-045 



4 

36-382 

8*720 



5 

38-094 

8-710 




The first column in Table 2 has been compiled according to 
eqns (15), the second and third have been calculated schemati¬ 
cally according to Table 1. The fourth column containing ele¬ 
ments K 2 ] C has been obtained by means of relation (19). 

The elements of the remaining two rows of matrices [K rs ] 
and [R r0 ] are calculated in a similar way. As [K rs ] is a symmetri¬ 
cal matrix, it is sufficient to calculate only its elements lying to 
the left of the main diagonal and those on the diagonal itself. 
The correctness of the calculation is checked by substituting into 
relation (20) which is the means of checking almost all numerical 
operations represented in Table 2 including the compilation of 
the first column r F Jc . 

By this method it has been possible to obtain a system of 
linear equations for the sought coefficients pertaining to the 
weighting function q (t) — 1: 

1-697 1-630 13*271 IdA [0*380“ 

1-630 3-260 2-890 D 2 = 0-694 

1-327 2-890 4-217J |_D 3 J [0G04_ 

As the elements of matrices |X rs ] and lF r0 ] are independent of 
the chosen degree L of polynomial D(z) the mere reduction of 
the respective matrices will suffice to meet the case of L = 1,2. 
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By the solution of the above system of equations coefficients D ( 
are obtained for i # 0, while the coefficient D a follows from 
condition (8) 

A> = i-E A 

i-l 

In this way the following results have been obtained 


L 

A 

A 

A 

A 

3 

0-758 

0-046 

0*140 

0*057 

2 

0*768 

0-038 

0*194 


1 

0*776 

0-224 




In order to obtain the system of equations for the coefficients 
A pertaining to the weighting function q{t) = 1 (t- T ) it will 
suffice, in accordance with relation (21), to add to each element 
of matrices \K r ^\ and [i? r o] respectively the following quantities 


\K= - 0-4548 and A R = - 0-0646 


obtained by the numerical solution of eqns (22) and (23). In this 
way one obtains 


L 

A 

A 

A 

A 

3 

0*611 

0*186 

0*120 

0*084 

2 

0*631 

0*170 

0*199 


1 

0*642 

0*358 




The pulse-transfer function of the continuously acting mem¬ 
ber of the system G (z) = B (z)/A (z) and the polynomial D (z), 
the coefficients of which have just been calculated, determine 
completely the necessary transfer function (9) of the computer. 
The respective curves of the controlled variable * following the 
unit-step change of input signal w are represented in Figure 3 
for the weighting function q(t) = 1, and in Figure 4 for the 
function q (/) = 1 (t — T). 
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It can be seen from Figures 3 and 4 that, compared with the If the integral of squared error (10) has a minimum value, 


minimum number of steps (L = 0), a considerable improvement 
has been attained, especially in the case where in the minimal iza- 
tion of the integral of squared error the errors have been con¬ 
sidered as occurring only after the first sampling period. 

Derivations and Proofs 

First of all it will be proved that the above stated results hold 
for the case where all the poles of transfer function S(p ) are 
different from zero. 

The sequence of the increments of the variable e 2 * (t) 

Ae 2 [q = et(iT)-e$(iT-T ) 

has, according to eqns (5) and (6), the z-transform of 

& {Ae 2 [«]} -(1 ~z~ x )E 2 (z)=^D (z) A (z) (31) 

From this z-transform it follows obviously 

Ae2 ^ = b(T) s ? 0 D ° Ai -° (32) 

where A k = 0 for k < 0 and k > n; A <? 2 [/] — 0 for / > n + L. 
Eqn (32) contains all the X -f 1 coefficients of polynomial D (z); 
however, only L of them can be selected, as it is necessary to 
fulfil condition (8) that is D (1) = 1. For the purpose of fulfilling 
this condition let coefficient D 0 be detached 

00 = 1- Z^s (33) 

S- 1 

and eliminated from eqn (32) 

& e 2 [/]=—^] Ys^DsiAis — Ad + Ai (34) 

Now, the time curve of the manipulated variable y ( t ) is dis¬ 
solved into the sum of unit-step functions 

y(0= Z Ae 2 [i]l(t-iT) 

i — 0 

and the curve of the controlled variable * (/) can then be repre¬ 
sented by the superposition of the unit-step responses 

X(0=Z Aej [i] sa-iT) (35) 

i = 0 

Then for error e 1 (t) it holds that 
(0 = 1 - * (0 = 1 - "Z Ae 2 [i] s(t-i T ) 

i-0 

= Z A e 2 [f]s(t-iT) (36) 

i — 0 

where 

s(t-iT) = s(co)-s(t—iT) 

t>iT,s(t-iT)=- Z C v e p “ ( ' -,r) 

V — 1 

t<iT,$(t — iT) = C 0 (37) 


the coefficients of polynomial D (z) must fulfil the equations 

'-‘• 2 . l « 

According to the above indicated derivative of the integral it 
follows 

2j%«ei(0~df = 0 (39) 

The necessary partial derivative is determined from relations 
(36) and (34) 

By substituting (40) and (36) together with (34) into condition 
(39), and by altering the sequence of addition, it follows 

*00 (L n + L n+L 

<?(0) Z D s Z Z ( A i-s~ U (Aj-,. — Aj)s(t— i T) 

J 0 ls~l i — 0 j -0 

s (t -jT ) + "Y Y A l (. Aj _, -Aj) s (i t - iT)s(t - jT)} dt = 0 
i= 0 j = 0 J 

(41) 

Under the accepted pre-condition of transfer function S (p) 
having all its poles in the left semi-plane the integrals 

<r tJ -q(t)s(t-iT)s(t-jT)dt (42) 

converge and in eqn (41) the integral of the sum can be expressed 
as the sum of the integrals 

L n + L n +L 

Z D„ Z Z 

s= 1 i — 0 j — 0 

+ Z Z M'Aj-r-Aj)(F t j = 0 

1 = 0 j — 0 

and in the abbreviated form 

Z D s K rs + R rO = 0, r = l,2,...,L (43) 

S- 1 

with the following denotations 

K rs = Z "z C7 y (44) 

i = O 7 = 0 

= z "z (4 ; .- r -^) «7y (45) 

1 = 0 7 = 0 

For the degree L of the selectable coefficients D s the system 
of linear equations is thus obtained that can be written in the 
matrix form (11) as 

mm=-[«,<>] 

By interchanging the subscripts in relation (44) it can be 
easily proved that [isT rs ] is a symmetrical matrix. 
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By the solution of integral (42) it follows for the case of 
weighting function q (t) = 1 

a ij = c min (i,j) + C 0 Y P^-d(\j-i\) (46) 

v=l Pv 

where the function 6 (, k ) is determined by relation (13), and c 
according to relation (16). Integral (42) for the weighting func¬ 
tion q (t) = 1 (t — T) is to be denoted by It holds 

z' + O, 7+0, j — (jfj c 

^ = ^o = C 0 i ^-Z v -0(k) (47) 

v= 1 Pv 
n 

a oo — ~ Z C v z v S v 

v=l 

where <? v is determined by the second of relations (23), and 
z v = e^L 

The calculation of the elements of matrices [K rs ] and [R r0 ] 
according to relations (44) and (45) would be very laborious. 
For this reason let some arrangements be introduced that will 
simplify this calculation considerably. 

First, let us divide relation (44) into two terms 

n + Ln + L 

K rs =Y X A^Aj.-Aj) 

i — 0 J = 0 

“EE MAj-r-A^aij 

1 = 0 7 = 0 


If we denote 


r Ui + s= Z Aj(CT i + St j + r <?i + s,j) 

7 = 0 


we obtain relation (18) 


R rs = X Ai r U i+s 

i = 0 


(50) 


(51) 


All values of r U required for the calculation of the rth row 
of matrices [K rs ] and [R r0 ] can be obtained as the product of the 
rectangular matrix 

Eqn (52) * 


and of the column matrix 




It will be proved that in the case of weighting function 
q (?) = 1 matrix [cr] has all its elements lying on the lines parallel 
to the main diagonal of the same value. For the /nth element of 
the /rth parallel above main diagonal it holds 

" C 

ffm,*+r+m-V*+m = c/n + C 0 X -~—0(k + r) 

V = l Pv 




= 0(k)-0(k + r) 


Now, if the summating subscript i in the first member is shifted 
by s 9 i.e. i — s —> z, and considering that A. t — 0 for i > n and 
i < 0, it follows 

K rs= X E M A j-r-Aj)a- i + s ,j 

i = 0 7 = 0 

-EE AO*;-,-AM; 

i = 0 7 = 0 

According to (45) the second term equals i? r0 , and let the first 
be denoted 

R rs = E A X (A-r-AOo'i+s.j ( 48 ) 

i = 0 7 = 0 

In this way relation (19) has been obtained 

K r =R rs ~R r o (49) 

As the term i? r0 represents a special case of R rs with 5=0, 
it will suffice further to seek only the numerical solution of (48) 
for R rs . Let it be written in the following form 

n n 

Rrs Z A-i Z Aj ip^i+s, j + r &i + s,j) 

i=0 7=0 


As this relation is independent of m, all elements lying on 
this parallel are equal, and they may be denoted by the same 
symbol 

r r k = 0(k)-0(k + r) 

Similarly it holds for the elements on the /cth parallel below the 
main diagonal 

r r k =a k +m,r+m-^k+m,m=c mm(lc, r )+ 0(k)- 0(\r- k\) 

For the main diagonal k = 0. 

Due to this property of matrix [a] it is possible to arrange 
the numerical solution of matrix product (51) into a scheme 
shown in Table 1 (a) which can be easily found by comparing 
both methods of calculation. 

It remains yet to prove the validity of formulae (21), (22), 
and (23) by which the former results are to be corrected, if errors 
are being considered only after the first sampling period. By 
substituting into eqn (50) for a i;} (46) the terms (47) calcu¬ 
lated for the weighting function q (j) = 1 (t — 7), it can be seen 
that only the first column has been altered. Obviously it holds that 

r U k — r U k + ( ( ?k, r~~ G k,r~Gk, O + 0 M 0 (54) 


* Eqn (52): 


G 0,r 

0Q, 0 J 

^O, 1+r 

~ a o, ll 

0Q, 2+r 

^O, 2 ? 

• ^O, n + r 

— 00, n 



a Ur 

i 

o 

G l, 1+r 


a l, 2+r 

01, 2 5 

• a l,n + r 

-01,n 


W = 

°2 ,»• 

~ a 2, 0^ 

02 , J + r 

— 02,11 

<J 2, 2+r 

1 

C) 

K> 

to 

* a 2, n + r 

-02 ,n 



J*n + L t r 

~~ G n + L, 0 5 

a n + L, 1+r 

— 0n + L,ll a n + L,2 + r 

“ °Vi + L, 2 5 • 

** a n + L,n + r 

0n + L, n 


( 52 ) 
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When calculating the term in the parentheses, it is necessary to 
differentiate two cases: k > 0 and k = 0. By substituting 
relations (46) and (47), we obtain in the first case the relation 

n C 

k>0,a k r — (j kir — d k) o J tcr kiO = Co —~(1 —z v ) = /c 

v=l Pv 

which is independent of k. Similarly for k = 0 


cr o,>*~~ <T o,r"” cr o, o + ^o, o 

= -c 0 z <±(i-z t )+ i c v zA+c 0 E —0(o)= k„ 

v=l Pv v = 1 v = l Pv 


The checking formula (20) can be verified by substituting 
relations (44) and (45) and by using the relation 

4- r, j + r ^i, / ^ 

that follows from eqn (46). 

In Case (£>), with the transfer function S(p ) having one zero 
pole, the continuously acting member of the system is astatic 
[s (oo) = oo], and integrals (42) are not converging. It is possible 
to by-pass this difficulty, if the curve of the controlled variable 
is not represented as the superposition of unit-step responses, 
but as the superposition of responses to rectangular pulses. 
Otherwise the procedure of derivation is the same as in Case (a). 


With this notation the relation (54) may be rewritten in the References 

f° rm ^ 1 Ragazzini, J. R. and Franklin, G. F. Sampled-data Control 

k> 0, r U k = r U k + KA 0 Systems. 1958. New York; McGraw-Hill 

„ (55) 2 JurYj E. I. Sampled-data Control Systems. 1958. New York; 

k = 0, Uq = U 0 + k 0 A 0 Wiley 

3 Tou, J. T. Digital and Sampled-data Control Systems. 1959. New 
For the verification of formulas (21) and (22) it will suffice York; McGraw-Hill 
to execute operations (51) and (49) with the relations (55), and 4 Strejc, V. Ensuring reliability in complex automation by auto- 
to denote 7v 0 — k = X. matic digital computers. Automatisace. V (1962) 5 
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Oscillations Sous-harmoniques dans un Asservissement 

par Plus-ou-moins 

J-C. GILLE, S. WEGRZYN et J-G. PAQUET 


Summary 

Conditions for the existence of /?th-order subharmonics in an on-off 
control system whose linear part is of any order are established. In the 
case of a symmetrical relay without a dead zone use of Hamel locus 
of the system (instead of Tsypkin locus) immediately shows that only 
odd subharmonics can occur and provides a geometrical interpretation 
of commutation within half-period. It is thus seen that, for conventional 
servomechanisms, subharmonics, especially of high order, seldom 
appear, but the presence of backlash makes them more apt to appear; 
some particular types of systems easily generate subharmonics of all 
orders. Use of Hamel loci makes immediate generalization possible 
for the case in which a proportional-plus-derivative compensator is 
placed before the relay. Investigation of the global stability, or stability 
in the large, or the oscillations by direct simulation verifies these 
results. It shows that the border between two possible oscillations in¬ 
volves hysteresis, i.e., there is frequency entrainment; the part played 
by commutation within half-period appears as the cause of the dis- 
appearance of the higher-order subharmonic. Conditions for existence 
in the presence of asymmetry or of a dead zone are obtained as an 
extension of the Tsypkin method for determining forced oscillations 
from two families of loci. In the case of a non-symmetrical relay the 
problem is shown to be solved with one locus when the linear part of 
the servo system has integration or has a zero equal to zero. 

Sommaire 

On donne des conditions necessaires d’existence pour le sous-harmo- 
nique d’ordre n d’un asservissement par plus-ou-moins dont la partie 
lineaire est d’ordre quelconque. Dans le cas d’un relais symetrique sans 
seuil l’utilisation du lieu de Hamel (au lieu de celui de Cypkin) montre 
immediatement que seuls peuvent exister les sous-harmoniques d’ordre 
impair; elle donne Interpretation geontetrique du phenomene de 
commutation prematuree. On voit ainsi que pour les asservissements 
usuels les sous-harmoniques, surtout d’ordre eleve, apparaissent diffi- 
cilement, mais la presence d’hysteresis favorise leur production; cer¬ 
tains types particuliers de systemes presentent facilement des sous- 
harmoniques impairs de tous ordres. Grace au lieu de Hamel ces resul- 
tats se generalised sans calcul pour le cas oil un correcteur derivd 
precede le relais. L’etude de la stability globale des oscillations par 
simulation vdrifie ces resultats: elle met en evidence une hysteresis 
dans la frontiere entre deux oscillations possibles, i.e. l’existence d’un 
trainage de frequence, et le role physique joue par la commutation 
prematuree dans la disparition de l’harmonique d’ordre plus eleve. 
Les conditions d’existence dans le cas dissymetrique ou avec seuil 
s obtiennent par une extension de la methode de Cypkin de determina¬ 
tion de 1 oscillation forcee par deux families de lieux. Dans le cas d’un 
relais dissymdtrique avec une fonction lineaire possedant une integra¬ 
tion ou un zero nul on montre qu’on peut rdsoudre le probleme avec 
un lieu unique. 

Zusammenfassung 

Es werden die notwendigen Bedingungen fiir die Existenz einer w-ten 
subharmonischen Schwingung eines Zweipunktregelsystems, dessen 
linearer Teil von beliebiger Ordnung ist, angegeben. Fiir ein Relais 
mit symmetrischer Kennlinie ohne Totzone (Ansprechempfindlich- 
keit) zeigt die Benutzung der Methode nach Hamel (nicht nachZypkin) 


sofort, dab nur subharmonische Schwingungen von ungerader Ord¬ 
nung auftreten konnen; daraus bekommt man auch eine geometrische 
Deutung fiir das Umschalten wahrend der ersten Halfte der Periode. 
Es zeigt sich, dafi bei iiblichen Regelkreisen subharmonische Schwin¬ 
gungen, besonders hoherer Ordnung, selten auftreten; Hysterese hin- 
gegen begunstigt deren Auftreten. In einigen besonderen Arten von 
Zweipunktregelsystemen treten leicht subharmonische Schwingungen 
von beliebiger ungerader Ordnung auf. Die Benutzung der Methode 
nach Hamel lafit diese Ergebnisse ohne Berechnung fiir den Fall ver- 
allgemeinern, dafi ein PD-Kompensationsglied vor dem Relais liegt. 
Untersuchungen fiber die Stabilitat im Grofien mit Hilfe eines Analog- 
rechners bestatigen diese Ergebnisse. Es zeigt sich, dafi in dem Bereich 
zwischen zwei moglichen Schwingungen eine Hysteresewirkung be- 
steht, die eine Frequenzmitnahme erzeugt. Durch Umschalten wah¬ 
rend der ersten Halfte der Periode verschwinden die subharmonischen 
Schwingungen hoherer Ordnung. Bedingungen fiir das Auftreten sub- 
harmonischer Schwingungen bei einer asymmetrischen Relaiskenn- 
linie oder bei einem Relais mit Ansprechempfindlichkeit erhalt man 
durch Verallgemeinerung des Zypkinschen Verfahrens zur Bestim- 
mung von erzwungenen Schwingungen durch zwei Ortskurven- 
scharen. Im Falle eines asymmetrischen Relais zeigt sich, dafi das 
Problem mit Hilfe einer einzigen Ortskurve losbar ist, wenn der 
lineare Teil des Regelsystems integrates Verhalten aufweist oder einen 
Pol im Nullpunkt besitzt. 


Introduction 

L’existence d’oscillations sous-harmoniques a fait l’objet 
d’etudes a cause de son importance technique, soit qu’on 
recherche ce phenomene pour realiser une division de frequence 1 , 
soit qu’on l’evite a cause de son influence defavorabie 2 ’ 3 . Plus 
rares, et recents, sont les travaux 4-8 consacres au cas des systemes 
asservis. Ci-apres nous presentons une etude de cette question 
pour les systemes asservis par plus-ou-moins. Nous considerons 
(Figure 1 ) un systeme a retour unitaire (entree e; erreur s) dont 



Figure J 
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la branche directe comprend un correcteur 1 + Ip, un organe 
non-lineaire N et une boite L ( p ) lineaire de degre arbitraire. 
On sait 9jl ° que tous asservissements dont A est le seul organe non- 
lineaire, situe dans la chaine directe, peuvent se ramener a ce 
type. La boite N represente un element par plus-ou-moins 
(ci-apres appele «relais») a caracteristique symetrique (w = 
M sign x) ou non (w = + ou - M 2 ), avec seuil ± A/2 et 
hysteresis ± hjl . 
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On sait 9 ’ 10 que lorsqu’un tel systeme est force par une entree 
periodique d’amplitude F et frequence co f il y a un seuil de 
synchronisation F f qui depend de co f : une oscillation forcee 
fondamentale de frequence co f n’existe que si F > F f . Le present 
travail contribue a etudier le comportement du systeme dans 
la zone (F < F f ), ombree sur les Figures 3 , 4, 5, ou il ne peut 
y avoir d’oscillation fondamentale, en y etudiant la possibility 
d’oscillations sous-harmoniques de frequence co f /n, n etant 
entier. Il suppose connues les methodes developpees indepen- 
damment par Hamel en France et Cypkin 11 en U.R.S.S. dont 
il existe des exposes elemental'res 9 * 10 indiquant les references 
originates. 


Sous-harmoniques dans le Cas Symetrique sans Seuil 

(A = 0 5 M x = m 2 ) 

On sait 11 que le seuil de synchronisation F f pour la fon¬ 
damentale est donne lorsque A — 0 par 



V{(a f )+~ K(3co / )+yF(5co / ) + ... + a 
V(oS) = Im L(joj) 


4 M 

71 


Sakawa 6 et les presents auteurs 7 ’ 8 ont montre facilement que le 
seuil de synchronisation F n pour le sous-harmonique d’ordre n 
est de meme donne par 



+ 1 v(^) + 
3 V n 


1 

5 



( 1 ) 


Dans ces expressions <x n (ou oc) est l’ordonnee du point de 
frequence n co f (ou co f ) du lieu de Cypkin 9 ’ 10 du systeme. 

D’ou la technique suivante pour trouver des conditions 
necessaires de production du sous-harmonique d’ordre n: 

(1) tracer la courbe F f versus log co f , ombrer la zone sous-jacente; 

(2) la courbe F n versus log co f s’obtient en la decalant vers la droite 
de log 2 n octaves (1.6 octave pour n = 3, 2.3 pour 5, 2.8 pour 7, 
etc.); (3) alors le sous-harmonique d’ordre n peut s’observer dans 
la partie de la zone ombree situee au-dessus de la courbe F n . 

Nous croyons qu’il y a avantage a raisonner geometriquement 
dans le plan ( e , 8 ), en utilisant le lieu de Hamel et non le lieu 
de Cypkin. On est amene a couper la droite de commutation D 
(e -j- X s = — h/2) par le lieu S n deduit de la courbe fermee de 
representation parametrique e (t), e (r) par la translation qui en 
amene le centre au point co n = oo f ln du lieu de Hamel H 
(Figure 2). Ce point B n a pour coordonnees 



U(co) = RoL(jco) 


Il n'y a sous-harmonique que si coupe D (et on montre 
facilement comme pour la fondamentale 11 qui seule est stable 
Toscillation qui correspond a V intersection la plus bas situee). 


Cette fa<?on de raisonner presente les avantages suivants: 

(1) L’inspection geometrique permet immediatement une 
discussion qualitative, notamment de prevoir (J) les fonctions 
L O) susceptibles de donner des sous-harmoniques — celles dont 
un segment important du lieu de Hamel, parcouru dans le sens 
des frequences croissantes, s’eloigne de la droite de commuta¬ 
tion — et (//) le r61e favorisant de Fhysteresis. 

(2) Sur la Figure 2, le point B n decrit un lieu R n facilement 
deduit de L ( p ), et le point C n [e (/), 8 (/)] un lieu P n qui se 
construit par composition de mouvements, comme explique 
ailleurs 7 ’ 8 par les auteurs. C' n represente une commutation 
de — a -j- et C n est la commutation a mi-periode. On voit que 
B n Qet B r n C' n sont paralleles et necessairement de sens oppose, 
done n est impair: il n’y a pas de sous-harmoniques d’ordre pair 
dans le cas symetrique. 



Figure 2. Production du sous-harmonique d'ordre n, cas symetrique. 
Le lieu P n ' montre une commutation prematuree 


(3) Lorsqu’il y a un correcteur (X > 0) la methode se genera¬ 
lise sans aucun nouveau calcul: le seuil F n s’obtient en ecrivant 
que $ n est tangent a D. Dans le cas d’une entree harmonique 
e (t) = F sin co f t , le lieu S n est l’ellipse 


cq} (e - a) 2 + (e - fi) 2 = cojF 2 
d’ou l’expression explicite de F n 



[(fe/2 + a) Xcjo 2 f —p~] 2 

coj(l +X 2 co 2 ) 


( 2 ) 


dont ( 1 ) est un cas particulier. 

(4) On possede une interpretation geometrique de la condition 
supplementaire qu’il n’y ait pas de commutation avant la demi- 
periode. Sur la Figure 2 la courbe P n ' coupe la droite de commu¬ 
tation de faqon prematuree en C n " avant la demi-periode (pour 
t < n 7i J to f ). On conpoit que les harmoniques d’ordre eleve, 
dont les courbes P n ' ont plusieurs points doubles, presentent fa¬ 
cilement ce phenomene quand on agrandit i- e * augmente 
l’amplitude de l’entree. On en verra 1’interpretation plus loin. 


Exemples 

{a) Soit d’abord le systeme regulier L(p) = l/p (1 + p) 
(1 + ap). La courbe F f versus log co f a la forme indiquee par la 
courbe C x des Figures 3 ou 4. 

En l’absence d’hyster 6 sis (Figure 3) des oscillations sous- 
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harmoniques de tous ordres sont possibles mais seulement dans 
des zones etroites du plan (co f , F). Le sous-harmonique d’ordre 3 
apparait dans la zone hachuree horizontalement; celui d’ordre 5 
(et de meme les suivants) n’apparait que dans le tout petit 
triangle oil la courbe C 5 passe en-dessous de C l5 c’est-a-dire 
pour une entree de tres faible amplitude et dont la frequence a 
justement ete choisie extremement voisine de 5 co 0 . Lorsque 
a 0 toutes les courbes C n sont au-dessus de la courbe C x : les 
sous-harmoniques n’apparaissent pas. 



Figure 3. Sous-harmoniques Fun systeme regulier L (p) — 
\/p (1 + p) (1 + ap) parplus-ou-moins sans hysteresis 



Figure 4. Uhysteresis facilite Vapparition des sous-harmoniques 



Figure 5. Sous-harmoniques du systeme particulier L (p) = 
p/( 1 + P ) (1 + ap) 


La presence d’hysteresis facilite l’apparition d’oscillations 
sous-harmoniques, car le lieu de Hamel decrit dans le sens des 
frequences croissantes s’eloigne de la droite de commutation 
sur un segment plus long. On peut le verifier.aussi directement 7 * 8 . 
La Figure 4 est relative au meme cas (a = 0.1) que la Figure 3. 
On constate de plus larges zones de possibility pour les sous- 
harmoniques d’ordre 3 (hachures horizontales), 5 (verticales), 
7 (obliques), etc., leur aire decroissant lorsque n augmente. 

(b) Notre methode permet d’etudier egalement le systeme 
particulier L(p) = p/(l + p) (1 + ap) avec relais inverse. Les 
courbes F f et F n versus log co f sont alors celles de la Figure 5 


( a = 0.1). Pour une frequence co f telle que celle indiquee sur la 
figure on peut avoir des sous-harmoniques de tous ordres, ceux 
d’ordres eleve etant obtenus pour de faibles valeurs de F. Pour 
des frequences d’entree plus hautes on ne peut produire que les 
sous-harmoniques d’ordre eleve. 

11 est facile de retrouver directement ces resultats {Figure 6, 
ou on a suppose a = 0 pour rendre plus clair le mecanisme de 
generation des sous-harmoniques). On peut les expliquer par 
la forme du lieu de Hamel du systeme (la meme que celle des 



Figure 6. Oscillations sous-harmoniques d'ordres 3 et 5 du systeme 
particulier L (p) — p 1(1 + p) 



Figures 7. Simulation sur calculates analogique du systeme 
L (p) == 4/p (1 +0-1 p) 


lieux de la Figure 10) car dans toute une bande 0, <o m ce lieu 
s’eloigne de la droite de commutation, d’ou la possibility de 
tous sous-harmoniques. 

Cas ou Plusieurs Oscillations sont Simultanement Possibles 

II peut arriver qu’a une frequence a) f deux sous-harmoniques 
d’ordres m et n(m < n) soient possibles (F > F m et F n ). La 
theorie ci-dessus — comme la plupart des theories sur l’existence 
d’oscillations forcees — donnant seulement des conditions 
necessaires d’existence, nous avons recouru a la simulation 
(montage du type de la Figure 7) pour savoir lequel s’observe 
effectivement, c’est-a-dire trouver les conditions egalement 
suffisantes. 

(a) Si JP m < F n (co < co l sur la Figure 8 ), nous avons toujours 
observe le sous-harmonique d’ordre le moins eleve m. 
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(b) Si F m > F n (cas oj > co t ), la frontiere presente une sorte 
d’hysteresis. (1) Lorsqu’on fait decroitre l’amplitude d’entree F 
on observe d’abord le sous-harmonique m, et le sous-harmonique 
n apparait exactement pour Fegal a la valeur F m qu’indique la 
theorie, soit sur la courbe C m . (2) Si au contraire, on franchit C m 
de bas en haut on peut observer au-dessus de C m l’un ou l’autre 
sous-harmonique, selon les conditions initiales du probleme 
(cette dependance est vraisemblablement complexe; experimen- 


coupent determinant le cj) et le y qui caracterisent le sous-har¬ 
monique d’ordre n. 

Nous avons calcule et donnons en Figure 9 ces families de 
lieux pour L(p) — 1/(1 + p) (1 + ap) avec a = 0,1; les lieux 
y — 0,5 sont les lieux de Hamel pour l’oscillation symetrique 
translates horizontalement de (M 1 — M 2 )/2. La Figure 10 
montre les families de lieux pour le «systeme particulier» 
mentionne ci-dessus L(p) = p/( 1 + p) (1 + ap) avec a = 0,1. 



Figure 8. Frontiere entre les oscillations d’ordres m et n quand on fait 
croitre ou decroitre I'amplitude d'entree F: trainage de frequence 


talement on constate que la probabilite d’apparition du sous- 
harmonique d’ordre le plus eleve it decroit tres vite quand on 
augmente F) ; le passage au sous-harmonique d’ordre mo ins eleve 
m s’effectue par Tapparition d’une commutation prematuree qui 
fait disparaitre le sous-harmonique d’ordre n. Ce «trainage de 
frequence» s’observe sur les exemples des Figures 3 et 4. II peut 
mettre en jeu les sous-harmoniques d’ordres 3 et 5 , ou 5 et 7 , etc.: 
dans ces cas il est specialement facile a mettre en evidence sur le 
systeme de la Figure 4. II existe egalement dans le cas ou m = 1: 
le seuil de synchronisation du fondamental F f presente lui aussi 
ce caractere d’hysteresis. 

Au total, cette premiere approche du probleme de la stabilite 
globale (conditions necessaires et suffisantes) des oscillations 
forcees met en evidence les resultats suivants: 

( 1 ) II existe un trainage de frequence dans les systemes par 
plus-ou-moins. 

(2) A ce point de vue toutes les oscillations forcees (fondamen- 
tale et sous-harmoniques) jouent le meme role. Done dans la 
zone F > F f9 ou on admettait habituellement qu’existe la fonda- 
mentale, les choses sont en realite plus complexes et on peut 
parfois observer un sous-harmonique quand on opere en faisant 
croitre F. 

(3) Le phenomene de commutation prematuree etait jusqu’ici 
considdre 12 comme une possibility theorique guere rencontree 
dans les systemes reels: nous connaisons maintenant sa significa¬ 
tion physique — la disparition d’un sous-harmonique au profit 
d’un autre d’ordre moins eleve — et comprenons ainsi pourquoi 
son inter§t reel n’a pu apparaitre avant qu’on se penche sur le 
probleme des sous-harmoniques. 

Cas d’un Relais a Caract&ristique non Symetrique ou avec Seuil 

Lorsque le relais a une caracteristique dissymetrique (w= -f M 1 
ou — M 2 ) les deux «demi-periodes)) 7 ^ et T 2 du sous-harmonique 
(n + T 2 = 2 Ttn/cof) sont inegales. Pour les determiner on 
peut 5 appliquer a la frequence a) f ln la methode proposee par 
Cypkin 11 pour la fondamentale. Cela conduit a centrer la courbe 
fermee S n aux differents points cof/n des lieux de deux families, 
et a en deduire par intersection avec la droite de commutation 
deux courbes du dephasage </> versus y = 7 ^/( 7 ^ + T 2 ), qui se 



Figure 9. Families de lieux de Hamel pour le systeme regulier 
L ( p ) = 1/(1 + p) (1 + 0-1 p) dans le cas d’un relais dissymetrique avec 
M x = 15 etM 2 = 10 



Figure 10. Families de lieux de Flamelpour le systeme particulier 
L (p) = p/( 1 + p) (1 + 0*1 p) dans le cas dissymetrique avec M 1 = 15 

et M 2 = 10 
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Le cas d’un relais avec seuil pent se traiter de la meme fagon 
avec Faide des families de lieux de Cypkin definies au para- 
graphe 23 de Cypkin 11 . 

Cette methode resoud theoriquement le probleme. Dans le 
cas dissymetrique on note que cette fois les vecteurs B n ' C n ' 
(' == Ti + r 2 ) et B n C n (t = T x ) pour y donne ne sont plus 
necessairement paralleles. On peut done obtenir aussi des sous- 
harmoniques d’ordre pair. De plus Finspection des lieux montre 
que la production de sous-harmoniques devient plus difficile a 
mesure que la dissymetrie s^accentue. 

Malheureusement la discussion plus precise de l’influence 
des divers parametres n’est guere possible a cause du grand 
nombre des courbes en jeu. Toutefois le probleme se simplifie 
dans les cas ou la fonction de transfert lineaire L(p) est telle 
Qu’on connaisse a priori une relation entre T ± et 7 2 . Cela se 
produit notamment lorsque: 

(1) L(p) possede une integration, c.-a.-d. le pole p = 0 (e’est 
le cas de la majorite des servomecanismes reels) — on a alors 
la relation T x /T 2 = M 2 jM x \ 

(2) L(p) possede le zero p = 0 (exemple: le systeme particu- 
lier etudie en (3)) — on a alors T ± — T 2 . 

Ci-apres nous developpons le calcul dans le premier cas, ou 
nous montrerons que le probleme de determination d’une oscil¬ 
lation forcee, fondamentale ou sous-harmonique, peut se re- 
soudre avec un lieu unique. 


Calcul dans le cas oil L(p) Possede une Integration 

Soil d’abord le cas tres simple ou L(p) = 1 Ip (1 4 7». En 
integrant . 

a partir des conditions initiates s 0 , s 0 lors de la commutation de 
— 4 -I- il vient, posant r t = exp(~ tJT) pour simplifier l’ecri- 

turc: e=s 0 -At+T(e 0 + A)(l-r) 


e = gr — A(l — r) 

Constant t x pour lequel e = — e 0 est donne par: 

8 = — s 0 = £q — At x 4- T (e 0 4-^4) (1 — ^i) 
On determine ainsi: 

-ISo + Att-ATil-rO 

e °" ru-ru 


(3) 


et 


eOi) = ^o + (2- £ 0 — )IT 

\ r 1 (~2s 0 -{~ At x ) — AT (l — r x ) 

£ ^ )= -T( i-7o 


De la meme faqon, Fintegration de 

Ts + b=+KM 2 = B 

a partir des conditions e x = — £ 0 et s x — £(4), donne 
s = 8 x 4 Bt 4 , T'(£i“5)(l-r) 



En prescrivant alors, pour la periodicite, s(/ 2 ) = £0 ( ce Qui re “ 
vient a appliquer a la droite de commutation D' de — a 4- la 
methode generate de la transformation ponctuelle 14 ’ 15 ), on trouve 
que les deux demi-periodes 7^ = t x et 7 2 = t 2 doivent satisfaire 
la relation annoncee _ _. , 

t x m x =t 2 m 2 


qui determine directement le rapport 3 = T 2 /T x des demi- 


periodes : 


5 = M x IM 2 


Les conditions de periodicite sont done les relations (3) a (5) 
qui, avec (6), determinent e 0 et £ 0 . La resolution de ce systeme 
transcendant peut se faire soit graphiquement, soit pour plus de 
precision avec calculateur (machine ANALAC, ou par iteration 
avec un calculateur digital). Pour chaque valeur de d (qui cette 
fois est connu) on construit ainsi (Figure 12, ou r = 0,1) un 
lieu unique £ 0 versus e 0 gradue en (T x 4 7" 2 ), qui donne direc¬ 
tement Foscillation libre et le sous-harmonique par intersection, 
avec la droite de commutation D' de — a -h, respectivement 
de ce lieu et de la courbe fermee (e, e) (ellipse si Fentree est 
harmonique) centree en son point co '= 2 njn (T x 4- 7 , 2 ). 

La methode est exactement la meme pour des fonctions L(p) 
avec integrations plus compliquees, telles que 1 Ip (1 4* Tp) 
(1 4- T'p). Les seconds membres des equations (3) k (5), obtenus 



Figure 12. Lieu de Hamel unique pour le systeme regulier simple 
L (p) — 4 Ip (1 4* 0*1 p) dans le cas dissymetrique M x = 6-25, M 2 = 12*5 

(fi = 0*5) 




Figure 11 


Figure 13. Lieu de Hamel unique pour le systeme regulier 
L(p) = 4 Ip (1 4 p) (1 4 0*1 p) dans le cas dissymetrique M x = 6*25 
M 2 = 12*5 (3 = 0*5) 


208 
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avec Faide depressions connues 16 , ont une ecriture plus com- 
pliquee, la quantite d est toujours donnee directement par 
l’equation (6) et pour chaque d on a un lieu unique gradue en to 
(Figure 13) a partir duquel les sous-harmoniques se determined 
comme dans le cas symetrique. Notamment, les seuils de syn¬ 
chronisation s'obtiennent en tragant versus log co f l’abscisse du 
lieu par rapport a la droite de commutation D' et en decalant 
cette courbe parallelement a l’axe des frequences. 

La presente etude a ete rendue possible grace a un genereux 
don du Conseil des Recherches pour la Defense {Canada). 
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Synthesis of Control Systems Operating Linearly for Small Signals 
and Approximately ‘Bang-Bang’ for Large Signals 

E. V. PERSSON 


Summary 

A non-linear controller operating linearly in the small-signal range 
and approximately ‘bang-bang’ in the large-signal range is proposed. 
Its application to a process comprising n integrators with limited nth 
derivative of the controlled output is treated. The non-linear feedback 
functions of the controller are given such a form that the response in 
principle is independent of the signal amplitude. The possibilities of 
giving the system a bang-bang character for large control deviations 
are investigated. 

Analogue-computer studies have been made on systems with 
limited second, third and fourth derivatives. These studies show that 
very satisfactory results are achieved for the large-signal response, 
which deviates only insignificantly from the bang-bang response. 

The proposed type of controller is very simple in application, since 
it is only necessary to make non-linear the normal feedbacks in a 
conventional controller. 

Sommaire 

L’auteur propose un regulateur non-lin&iire qui fonctionne lineaire- 
ment dans la gamme des petits signaux et approximativement «bang- 
bang» dans la gamme des grands signaux. II traite de son application 
a un processus qui comprend n integrateurs avec limitation de la 
meme derivee de la grandeur reglee. 

On donne aux asservissements non lineaires du regulateur une 
forme telle qu’en principe, la reponse soit independante de l’amplitude 
du signal. 

On examine les possibility de donner au systeme un caractere 
«bang-bang» pour de grands ecarts de reglage. 

Des dtudes au calculateur analogique ont dte faites sur des systemes 
avec limitation de la seconde, troisieme et quatrieme derivees. Ces 
etudes montrent qu’on obtient une reponse tres satisfaisante pour de 
grands signaux, reponse qui ne differe que d’une fagon insignifiante 
de la rdponse bang-bang. 

Le type de regulateur propose est tres simple a realiser en pratique 
de fait qu’il est seulement necessaire de rendre non-lineaires les reac¬ 
tions normales d’un regulateur classique. 

Zusammenfassung 

Es wird ein nichtlinearer Regler vorgeschlagen, der bei kleinen Signal¬ 
amplituden ein lineares Verhalten, bei groBen Signalamplituden 
angenahert Zweipunktverhalten aufweist. Seine Anwendung fur eine 
Regelstrecke, die aus n Integratoren mit beschrankter /iter Ableitung 
der RegelgroBe besteht, wird behandelt. Die nichtlinearen Riick- 
fuhrungen sind so ausgefuhrt, daB die Ubergangsfunktion im Prinzip 
von der Signalamplitude unabhangig ist. Die Moglichkeiten, dem 
Regelsystem bei groBen Regelabweichungen Zweipunktverhalten zu 
geben, werden betrachtet. 

Untersuchungen von Systemen mit beschrankter zweiter, dritter 
und vierter Ableitung am Analogrechner zeigen, daB sich sehr befrie- 
digende Ergebnisse erzielen lassen. Die Ubergangsfunktion bei groBen 
Signalamplituden weicht nur geringfiigig vom Zweipunktverhalten ab. 

Der vorgeschlagene Regler laBt sich in der Praxis leicht verwirk- 
lichen, da es nur notwendig ist, die normalen Rtickfuhrungen eines 
konventionellen Reglers nichtlinear zu machen. 


Introduction 

Conventional control systems designed as linear systems operate 
satisfactorily within the limits of linearity, and the desired 
response and accuracy can be achieved by means of correct 
design for which effective methods of synthesis exist. 

If, however, a system of this kind is subjected to such large 
changes in the command input or such large disturbances that 
limiting is reached, the response may be unsatisfactory and many 
systems will even become unstable. Thus a suitable compromise 
must be made between large-signal and small-signal response, 
a compromise which may mean that neither will be fully 
satisfactory. 

If, on the other hand, one designs the system for the fastest 
possible response with the control effort available, this will lead 
to a bang-bang system. Such a system will be very complicated, 
however, and difficult to apply if the process is described by a 
differential equation of higher order than the second. Further¬ 
more the bang-bang response is not desirable for small control 
errors, since very high-frequency oscillations will appear at 
steady state. 

It has been proposed 1 that the non-linear controller should 
be disconnected when the error has been corrected and that the 
steady state should be maintained by connecting up a new 
controlling device. It is hardly likely, however, that such a 
system would operate satisfactorily for small disturbances 
occurring frequently. 

This paper presents a system which operates mainly bang- 
bang for large control errors but linearly for small errors. The 
transition between the two modes of operation takes place 
smoothly without any switching devices. The author has aimed 
at a system which is simple to apply, and for this reason the 
requirements as to an exact bang-bang response have been 
waived. 

This paper is limited to the case where the control loop 
comprises a number ( n ) of integrators and where the input to 
the first integrator is limited to its absolute value, i.e. the nth 
derivative of the controlled output is limited. 

The Problem 


Let us start with a non-stabilized control system according 
to Figure 7, where it is assumed that the process is represented 
by n integrators. 



Figure 1. Non-stabilized system under study 
r 0 — command input; c 0 = controlled output; M — limiting level for c n 
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SYNTHESIS OF CONTROL SYSTEMS OPERATING LINEARLY FOR SMALL SIGNALS AND APPROXIMATELY ‘BANG-BANG’ FOR LARG E SIGNALS 


The following notation is used: 
r 0 = Command input 

r v = d v r 0 /d t v = vth derivative of the command input 
c 0 — Controlled output 

c v = Variable equal to constant • d v c 0 /d t v (see Figure 1) 
c n = Input to first integrator 

M = Limiting value for c n according to: — M < c n < M 
e = Input to the non-linear function representing the limita¬ 
tion 

K v (v = 1 ... n) — Gain of the integrators 

v-n 

k = n k v 

V = 1 

K 0 = Gain of the non-linear function within the linear range 
t = Time 

s = Laplace operator 

It is now desired to design a practicable non-linear controller, 
operating linearly for small signals and functioning satisfactorily 
even for large signals. The bang-bang response is taken as the 
ideal for large-signal response. 

The command input is assumed to have the form 


r 0 = b 0 + b l t+b 2 t 2 + ... b n _ l t n 1 (1) 

where b 0 ... b n ~ x are constants. As is apparent, it has been 
assumed that: 


d” 

'•„=—=<) 

d f 


( 2 ) 


Normalizing the Equations to Reduce the Number of Parameters 

The following variables are now introduced instead of the 
original ones: 


t = VK-t 

__d __1_ d_ = J_ 

P dx XfK d t~1jK S 


(3) 

(4) 

(5) 


dx 0 _c t K l —r l 
dx ~~ M\jK 


( 6 ) 


d(x v _i ) __ c v K i K 2 ... j: v _ 1 K v -r v 
dr ~ M [vT] u 


4 (Xi— j) 2 • • ■ — 1 _ £ n 

dx ~ M\XlKj ~M 


( 8 ) 

(9) 


With the new variables the block diagram is shown in Figure 2. 
x Q is the control error (disregarding scale factor) [see eqn (5)]; 
x v (v = 1 ... n) is the vth derivative of x 0 with regard to r; and 
the limiting value for x n is unity. 



Figure 2. Normalized system 

x 0 = control error in relative scale = c 0 — rJM; x v = d v xjd t v ; 


where r = y K t 

Since d n r 0 /dt = 0 according to eqn (2), all the variables x v 
at steady state will assume the value 0, and the behaviour of the 
system, according to Figure /, can be studied instead in the system 
shown in Figure 2 by allowing the variable x v to assume the 
initial value . . 

Ov)r = 0 

m[7kj v 

Introduction of a Non-linear Controller 

A non-linear controller is now introduced into the system 
shown in Figure 2. The variable y is determined by the control 
error x 0 and its derivatives x l9 x 2 ... x n ~ t with regard to the 
normalized time r. In order to make this controller practicable, 
y is assumed to be the sum of the single-variable non-linear 
functions f 0 (x 0 ), /i (x x )... f n ~ x (x^) as shown in Figure 3. 


1 



Figure 3. Introduction of non-linear controller with feedback functions f v 


The Large-signal Response of the System. Form of the Non-linear 
Feedback Functions 

With very large signals the non-linear function representing 
the limitation in Figure 3 is greatly overloaded with the output 
x n = + 1 dependent on whether y (the switching function) is 
positive or negative. 

If it is now assumed that it is possible to determine the non¬ 
linear feedback functions f v (x v ) so that the response will be 
that desired for a certain large signal level, e.g. for a certain 
initial value of x 0 , it is natural to require the same response for 
other large signal levels, though with altered amplitude and 
time scales. Which types of non-linear functions fulfil this 
requirement are investigated here. A further time transformation 
is now made for this purpose as follows: 


d 


( 10 ) 


where ft is a positive constant and q is a new operator deter¬ 
mined from . 

(ii) 
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The block diagram in Figure 3, due to the transformation (II), 
becomes that shown in Figure 4 , which in turn can be redrawn 
by means of amplitude transformations giving that shown 
in Figure 5. 

Comparing Figure 5 with Figure 3, it is apparent that the 
systems are exactly the same except that x v is replaced by 
& n ~ v x v and that the switching function y is replaced by y . If 
the non-linear feedback functions are now selected so that y will 
also be a positive constant times y (the constant may be a func¬ 
tion of /?), then $"- v x v as a function of r' and * v as a function 
of r will be indentical for the systems shown in Figures 5 and 3. 


1 



Figure 4. System with modified time-scale 


1 



Figure 5. System with modified time- and amplitude-scale 


The condition for this is consequently 


y'=4>Wy 

( 12 ) 

where 4> (fi) is an arbitrary, positive function of f$. 

The solution of (12) is 

\f v (x v )\ = k v \x v \ n ~ y 


Since / v (x v ) must be an odd function of x v , the final solution 
will be 


fv Ov) 



B 

|X V |"" V 


(13) 


In eqn(13) Jc v is a constant that can be freely selected for 
each v value, whereas B is an arbitrary constant that must have 
the same value for all values of v. 

Now that the form of the function is given according to 
eqn (13), the practical synthesis work can be accomplished 
without difficulty by aid of an analogue computer. This work 
may be limited to one large amplitude, and suitable values for 
the n constants k v can be determined. When a desired response 
has been obtained, the same response for other large amplitudes 
will automatically be obtained, although in another time scale. 


An attempt is now made, however, to determine the constants 
k v so that the response will approach the bang-bang response as 
closely as possible. 

Determining the Constants k v to Approach Bang-bang Response 

The bang-bang response comprises a series of time intervals 
with the duration r l5 t 2 ... n according to Figure 5, with x n 
varying between -f 1 and — 1. The numbering has been chosen 
so that x x refers to the last interval. At the end of this interval 
(t = r s ) the state of equilibrium, characterized by the fact that 
all the x v values are zero, is attained. 

In order to simplify the calculations a new normalized, but 
reversed time ft, is introduced according to 

£ = t s -t (14) 

which means that the final state for r = r s becomes the initial 
state for ft = 0, namely: 

x 0 =x x = ... =x n _ 1 = 0 for 3 = 0 (15) 
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Figure 6. Bang-bang response 


The variables x v are now related by the following equations: 



v = 0... rc —1 


Starting with x n according to Figure 6, interpreted as a sum 
of unit step functions, and applying successively eqn (16) gives 
the following value for x v at ft = ft m : 

1 T p-m-l 

^=(-i)"- v ^yr[%- v +2 i ( -i n& m -^ P r v 

(17) 

At this instant should change sign, which means that the 
switching function y should be equal to zero. 

The switching function has the form: 


v=n—1 v = n — 1 2 

y=- E /.W=- E K-, -rjlJCvF 

v = 0 v = 0 l-^vl 


(18) 


Inserting for x v in eqn (18) values x vrn according to eqn (17) and 
setting y equal to zero, yields the following condition for 
bang-bang response: 


y — n 1 


E r 




<KW 


•|^(5)|A=o (19) 


where 


m = 1,2, ...,n —1 


p = m~ 1 


^) = % _V + 2E 

P=1 


(-ir( 9 m - 9 p )'— 
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SYNTHESIS OF CONTROL SYSTEMS OPERATING LINEARLY FOR SMALL SIGNALS AND APPROXIMATELY ‘BANG-BANG’ FOR LARGE SIGNALS 


Equations (19) must be satisfied, by selecting the constants k v 
and 1 5, for all values of $ l9 # 2 ... ^n-i if the system is to operate 
bang-bang irrespective of the initial condition. This is not 
possible, however, except in the simple case n = 2. One is 
therefore restricted to requiring bang-bang response only for 
a step input. 

In this case the following applies, according to Burmeister 2 , 
for the relative duration of the intervals in Figure 6: 


rr sm 


2 


rn 
2 n 


for r = 1, 2 ... n — 1. 
From this: 


3,„ = 3„sin 2 


mn 
2 n 


( 20 ) 


Q p = 9„sm 2 -^- (21) 

In 

Using the relationships (20) and (21) and introducing the 
following function A of n , v and m : 

( 22 ) 


then eqn (19) can be transformed into 

v — n— 1 

E K(-iy 

v = 0 

m = l,2, — 1 (23) 


L0*-v)! 


j A(n,v, m) 

I A(n,v, m) | 


\A(n, v, m)| n “ v = 0 


p — m—X 

A(n,v,m) = 1+2 £ (-l)' 

p= i 


sm 


1 — 


2 P 71 
2 n 


sm 


* mn 
2 n 


The system of eqns (23) contains n — 1 equations, which 
must be satisfied by selection of the n constants k Q , k x ... k n - x 
and the constant B, and this is always possible. B and one of 
the constants k v can be arbitrarily selected. The system of eqns 
(23) determines, however, for a given value of B the relationship 
between the n constants k v . The calculation has been performed 
for n — 2, 3 and 4, the result being as follows: 


n = 2 



n = 3 


7c x _2 b/2 11 b/3 + 1 ] 


/C2_ J_ H B/3 —1 
/c 0 6 b/3 2 

When synthesizing according to eqns (24) to (26), the 
switching function will be zero at all those instants necessary 
for bang-bang step response. This does not necessarily mean, 
however, that the system must operate bang-bang, since it is 
possible that the switching function will pass through zero at 
other instants as well. An investigation of the derivatives of the 
switching function at the zero points suggests that the step 
response should be bang-bang for n — 2 and n = 3 irrespective 
of the value of B. For n = 4, the step response should be 


bang-bang for B < 2, but not for B > 2. When B > 2, the first 
switching is correct, as is also the beginning of the second one. 
The second switching is abnormal, however, with a number of 
rapid oscillations about zero taking place over a short interval. 

If the results are to be of practical use, it is naturally also 
necessary for the system to be stable; this is not certain even if 
the system has a bang-bang step response. 


n = 4 


kl 6 b ' 3([4( ^ 2+1) 

4 -1] b/4 + 1}(V2-1 ) b + 2 

nr i 

fco _ 24 B '* 2 

k 2 2 b/2 [4(V2+1) 4 
k 0 24 m 2 

k 3 1 + D 

,( v y) + w 2 -‘>'] 

-i] s/4 -i 

-nr 

k 0 24 b/4 

2 

Ki 1 ) +w5 - i) *] 



/ 

(26) 


Modifying the Non-linear Functions to give Linear Operation for 
Small Signals 

If the system in Figure 3 is to operate linearly within the 
small-signal range, the following must apply here: 


L(x,) = a v x v (27) 

where a v are constants selected by means of synthesis methods 
for linear systems. 

The following applies for large signals [see eqn (13)]: 

Mx v ) = k v £-\x v \^ (28) 

The constants k y are selected according to eqns (24)-(26) or 
from computer studies. Depending on whether Bjn — v is less 
than, equal to, or larger than unity, the functions will have the 
appearance shown in Figure 7 (a), (b) or (c). For the case 
Bjn — v — 1, k v has been selected equal to a v . In this case the 
function is the same in both ranges. For the other cases, where 
B/n — v =j= 1, the full-drawn curves in Figure 7 are selected. 
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When making this choice of the functions, the system will 
operate linearly for small signals and approximately bang-bang 
for such large signals that the linear range will be negligibly small. 
The transition between these two modes of operation takes 
place gradually without any need for switching devices. 

When selecting the non-linear functions according to 
eqn (28), the constant B in the exponent can be given any 
positive value. In the first place, however, a whole number ought 
to be selected within the range 1 < B < n. 

With this choice, one of the functions will be linear, namely 
fn~B (x n ~B) which is equal to k n ~B x n -B. The constant k n ~B is 
determined by 

k n ~B ~ Cl n ~B 

For practical reasons B = 1 is preferred, provided that this is 
consistent with satisfactory response. 

The choice of appropriate values for B and for the constants 
k v is best made with the aid of an analogue computer. Such an 
investigation has been started but not yet completed, and it is 
thus only possible to report the results obtained from preliminary 
investigations, which were intended to establish what possibilities 
exist to improve a system by introducing a non-linear controller 
of the type proposed. 


Preliminary Analogue Computer Studies 

Preliminary investigations have been performed on systems 
with two, three and four integrators (n = 2, 3 and 4). Some of 
the results are presented here. 

The synthesis in the linear range (selection of the constants 
a v ) has been made according to a method described in the work 
by Kessler 3 . This synthesis method will yield a system with a 
step response overshoot of about 5 per cent. 


Case n = 2 

In this case the system operates bang-bang for all initial 
conditions disregarding the small deviation due to the linear 
range in the non-linear functions. 

Some responses are presented in Figure 8, The feedback 
functions are as follows. 

In the linear range: 

a o = 2 ; f 0 (x 0 ) = 2x 0 

a i= 2 ’ f 1 (x 1 ) = 2x 1 

In the non-linear range: 

B= 1 

k 1 =a 1 =2 /iOi) = 2x, 

k 0 = 2 J2. [from eqn(24)] f 0 (x 0 ) = 2j2-^-\x 0 \ 112 

\ X 0\ 

The step response is shown in Figure 8 ( b ) and id) for two 
different amplitudes of the step. For comparison the correspond¬ 
ing step responses with a linear controller are shown in Figure 
8 (a) and (c). As is apparent, the non-linear controller operates 
almost ideally bang-bang for large steps. 

Case n = 3 

In this case the system is not stable with B = 1 and the 
constants k v selected according to eqn (25). With B = 3, how¬ 
ever, a system operating practically bang-bang is obtained. 
A few responses are shown in Figures 9 and 10 with the following 
feedback functions [not true for Figure 9 (d)]. 
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Figure 9. Step response for system with n = 3: (a) Linear controller; (b) non-linear controller; ( c ) non-linear controller; (cl) non-linear con¬ 
troller , but on a system of higher order than third 



Figure 10. Ramp response for system with n = 3; (a) Linear controller; (b) non-linear controller 
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In the linear range: 


a 0 = 8 

fo (x 0 ) — 8 x. 

a t = 8 

/i(xj = 8x 

a 2 = 4 

f 2 ( x f) = 4x : 


These values of the constants k v correspond to: 

^3 


fei 


-i=6-35 

fc 0 


k 2 


= 91 


k 0 


= 13-3 


whereas eqns (26) give for B — 6: 


In the non-linear range: 

B = 3 

fc o=a 0 = 8 /o(*o) = 8x 0 

ki = 16^/2 [from eqn(25)] (x„) = 16^2^|xj 3/2 

1*1 I 

/c 2 = y [from eqn (25)] f 2 (x 2 ) = y x] 


Figure 9 (a) shows the step response with a linear controller 
for such a large step that the stability has started to deteriorate. 
For an insignificantly increased step the system is quite unstable. 
Responses with the non-linear controller are shown in Figure 
9 (b) and (c) for two steps, both larger than the largest step that 
the linear controller can handle. As is apparent these responses 
are almost ideal bang-bang responses, which is not surprising, 
since the system has been designed for bang-bang step response. 
Figure 10 ( b ) shows, however, that the system operates practi¬ 
cally bang-bang even for other inputs. In this case, x 2 has been 
given an initial value, approximately equal to 1 (corresponding 
to a ramp input). Under the same conditions a system with a 
linear controller is unstable. The response with a linear con¬ 
troller and an initial value for x 2 immediately below the stability 
limit is shown in Figure 10 (a). 


Case n — 4 

In the case of four integrators, it became apparent that the 
design on the basis of eqns (26) does not yield a satisfactory 
response. The system is unstable for B — 2. For B = 4, limit- 
cycle oscillations were experienced. The same applies for .5 = 6, 
although the amplitudes of the oscillations were smaller than 
for B = 4. It is possible that a further increase in B would have 
made the system stable, but the investigation was directed 
instead towards finding a combination of constants /c v giving 
acceptable response for B = 6. 

The following feedback functions were selected: 

/o(x 0 ) = 4x 0 

ft (x 1 ) = 8x 1 
/ 2 (x 2 ) = 8x 2 

/ 3 (X3)=4x 3 

fo ( x°) = 4-8 |x 0 |3/2 

/i(x 1 ) = 30-5^L|x 1 | 2 
/ 2 (x 2 ) = 440x:] 

/ 3 (x 3 ) = 64 j—y|x 3 | 6 


In the linear range: 

a o~ 4 
a x — 8 

cl 2 — 8 
a 3 ~4 

In the non-linear range: 
fc 0 = 4-8 

k x = 30*5 
k 2 = 440 
k 3 = 64 



Although eqns (26) do not provide k v values that can be 
applied, they nevertheless yield values that do not differ from 
those selected more than by a factor of about 2. Equations (26) 
are therefore of value for selecting reasonably starting values 
when determining the constants k v in an analogue computer. 

A couple of step responses with a linear controller are shown 
in Figure 11 {a) and (b), the first one for a step that the system 
can just handle, the second one with a slightly increased step 
where the system becomes unstable. 

Corresponding step responses with the non-linear controller 
are given in Figure 11 (c) and (d). Figure 11 (e) and (/) show a 
couple of step responses with a considerably larger amplitude. 
A comparison of Figure 11 (e ) and (/) reveals that the responses 
have almost exactly the same form. 

In Figure 11 (/) the ideal bang-bang response has been 
plotted as the dotted line in the recording of the controlled 
output c 0 /M. Even if the system investigated for large amplitudes 
yields six switchings, while in an ideal bang-bang system only 
four switchings occur, the settling time is only about 50 per cent 
longer than the shortest possible. 

No investigations have been made on systems with limita¬ 
tions in a higher derivative than the fourth one. This does not 
mean, however, that the application is limited to fourth order 
systems. On the contrary, applications have been made on 
higher order systems with additional linear transfer functions 
introduced immediately ahead of the non-linear function 
representing the limitation (at <? in Figure 1). For example with 
three integrators behind the limitation function (n = 3), 
practically ideal bang-bang response is obtained even in this 
case, provided that the linear system is synthesized according to 
Kessler 3 and that a small correction is made to the constants k y , 
(This correction was obtained simply by assuming the limitation 
level M to be smaller than the real one by a factor of 1*4.) A step 
response for such a system is shown in Figure 9 (d). Thus the 
proposed type of controller can be used for a system of arbitrary 
order, as long as there are no limitations in derivatives of higher 
order than the fourth, and the transfer functions between the 
limitation and the controlled output are integrators. 

Conclusions 

The investigation reported in this paper shows that it is 
possible to design a non-linear controller, operating linearly for 
small signals and also having a satisfactory performance for 
large signals when used on a process, comprising a number of 
integrators, and in which one of the derivatives of the controlled 
output is limited. The controller is simple, since the only differ¬ 
ence in comparison with a conventional linear controller is that 
the feedbacks already existing in the latter are made non-linear, 
which can easily be achieved by means of biased diodes or non¬ 
linear resistors. The transition between linear and non-linear 
operation takes place smoothly without any need for switching 
devices. 
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SIGNALS AND APPROXIMATELY ‘BANG-BANG’ FOR LARGE SIGNALS 



Figure 11. Step response for system with n = 4: (a) Linear controller; (b) linear controller; (c) non-linear controller; Id) non-linear controller, 
(e) non-linear controller; (/) non-linear controller; Dotted curve: ideal bang-bang response 


An almost exact bang-bang response can be obtained for 
systems with limitation in the second or third derivative of the 
controlled output. In the case where the fourth derivative is 
limited, the response is sufficiently good not to justify a change¬ 
over to the extremely complicated controller required to achieve 
exact bang-bang response. By introducing the non-linear 
controller, improvements can also be achieved in the linear 
range, since the controller can be designed here for the best 
small-signal response without regard to the large-signal response. 
Even though the investigation presented here is confined to 
systems comprising solely integrators in the control loop, the 
results may be applied to cases where the integrators are replaced 
by transfer functions with poles sufficiently close to the origin 
in the s plane. It should therefore be possible to apply controllers 
of this type to a large number of processes. 

Remaining Work 

The studies are being continued along the following lines: 


(а) Further investigations with n =3 and 4, but with B =1 and 2 

(б) Investigation of systems with more than one limited deriv¬ 
ative. 

(c) Investigations with other types of transfer functions than 
integrators. 

(d) Investigation of how far it is possible to approximate the 
non-linear functions without losing the main advantages. 

The author expresses his thanks to Mr. Hj. Sorensen who 
performed the investigations on the analogue computer. 
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DISCUSSION 


M. Athans, M.I.T. Lincoln Laboratory, Lexington, Mass. U.S.A. 

It should be emphasized that the system of Figure 3 is not a time- 
optimal one. Since the word ‘bang-bang’ now implies time-optimal 
control it is somewhat confusing in this paper. I would like to ask the 
following questions. 

Is the system of Figure 3 stable for all possible initial conditions? 
How is B chosen? How can it be proved that the system is stable? 
Has the author investigated the relation of the response time to the 
minimum time? 

F. Mesch, Institut fur Regelungstechnik, T.H. Darmstadt, Germany 

At the end of the paper it is stated that the results obtained for the 
integrators might be extended to systems with poles sufficiently close 
to the origin of the s plane. I do not believe that this is true for complex 
poles. Furthermore, I feel it to be somewhat unrealistic to assume 
that the output signals of all the integrators can be measured in a 
practical system. 

E. Pavlik, Siemens, Lassallestr. 9, Karlsruhe, Germany 

According to Figure 3 of the paper, the non-linear controller feedback 
functions are taken from the outputs of the integrators of the con¬ 
trolled system. Normally, in technical control systems, these signals 
are not available. What means exist for approximated bang-bang 
control, when these feedback signals are not available? 

ML Hamza, ETH Zurich , Switzerland 

I would like to know how far it is possible to approximate the non¬ 
linear functions without losing the main advantages of this method ? 
At the end of this paper it was mentioned that such studies were being 
performed. I feel that this method will have many limitations for most 
systems in practice. From the previous discussion, I understand that 
studies were performed, using a system having complex poles. Would 
Mr. Persson please indicate how he would apply his method if the 
open-loop system transfer function is of the form 

K 

s (s 2 + 2 £a> 0 s + <Wq) 


E. V. Persson, in reply 

In reply to Mr. Athans’ remark concerning the word ‘bang-bang’, 
I refer to the title and the introduction of my paper, from which I hope 
it is clear that I am aware of the fact that the system is not a true 
bang-bang system (except for the case n — 2). In my opinion this is 
less important than the fact that the system can be made stable without 
having to compromise between small and large signal response. 

Investigations have indicated that the system of Figure 3 can be 
designed to be stable for all initial conditions, even if this has not been 
strictly proved. The system n = 3 and n — 4 were checked for stability 
over a large domain of initial conditions, as well as for harmonic 
response over a large frequency range. 

The choice of B is a compromise between practical considerations 
and the desire to approach bang-bang response as closely as possible. 
In my opinion B = 1 or 2 should be preferred, even if the ratio 
between the response time and the minimum time will increase for a 
fourth-order system from 1*5, as obtained in Figure 11 in my paper, 
to approximately 2. 

Mr. Mesch and Dr. Hamza have drawn attention to systems with 
complex poles in the ^ plane. Systematic studies of such systems have 
not been carried out, but the method has been successfully applied 
to a fourth-order practical system of this kind. Briefly, the method was 
applied by selecting the exponents in the non-linear functions from 
the analogue computer diagram simulating the system, neglecting the 
feedbacks existing between the integrators. Then the constants k v were 
determined by means of computer studies. 

Mr. Mesch and Dr. Pavlik remarked that normally all the quantities 
necessary for feedback are not available in practical systems. This is 
true even with a linear controller. To be specific, let us assume a 
process, consisting of three integrators, where no derivatives of the 
output are available. In order to make the linear system stable, one 
then has to produce the first and second derivative by means of lead 
networks. There are no additional difficulties in applying the method 
proposed in my paper. In fact, the method has been applied to a 
practical system of this kind. 

Finally, replying to Dr. Hamza’s first question, the non-linear 
functions can be approximated by three straight lines, without losing 
the main advantages. 
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Dual Input Systems with a Saturation Constraint 

R.S. GAYLORD 


Summary 

This paper extends the theory of dual-input systems to the problem of 
limiting the activity of an intermediate variable in the system in order 
to avoid saturation or excessive power consumption. The derivation of 
the general dual-input Wiener filter which includes the constraint is 
first outlined and discussed and then three degenerate cases are com¬ 
pletely solved. The Lagrange multiplier approach is used to handle the 
constraint mathematically. To demonstrate completely the concepts 
involved, a numerical example for a case is shown in detail. The nu¬ 
merical example demonstrates how one might ‘trade off’ or compro¬ 
mise in an actual system design in order to find the ‘best’ solution. 

Sommaire 

Ce travail etend la theorie de systemes a entree double au probleme de 
limitation de l’activite d’une variable intermediate d’un systeme afin 
d’eviter la saturation ou la consommation excessive de puissance. La 
derivation du filtre general de Wiener a entree double, contenant la 
restriction est discutee d’abord; ensuite, des solutions completes des 
trois cas deg6n6res sont presentees. On se sert de la m6thode du 
multipiicateur de Lagrange pour traiter le probleme d’une maniere 
mathematique. Pour presenter l’ensemble des concepts en question, un 
example numerique pour un cas particulier est donne. Cet example 
numerique montre comment on peut «echanger» ou arranger le calcul 
d’un systeme r6el afin de trouver la solution «la meilleure». 

Zusammenfassung 

Der Aufsatz erweitert die Theorie der Systeme mit zwei Eingangen auf 
folgendes Problem: In dem System soil die Auswirkung einer Zwischen- 
groBe begrenzt werden, urn Sattigung Oder ubermaBige Leistungsauf- 
nahme zu vermeiden. Die Darstellung eines Filters im Wienerschen 
Sinne mit zwei Eingangen, das die Beschrankungen cinschlieBt, wird 
zuerst abgeleitet sowie diskutiert und dann 3 (entartete) Falle voll- 
standig gelost. Zur Losung wird der Lagrange-Multiplikator verwen- 
det, um die Beschrankungen (Nebenbedingungen) mathemaiisch zu be- 
rucksichtigen. Ein in alien Einzelheiten durchgerechnetes Beispiel er- 
lautert eingehend das Verfahren. Das Zahlenbeispiel zeigt, wie man 
eine Abanderung oder einen KompromiB beim Entwurf eines wirk- 
lichen Systems treffen muB, um zur besten Losung zu gelangen. 


Introduction 

The problem of extracting the best estimate of a distorted signal 
by using more than one source of information is of considerable 
engineering interest and has been examined by various authors 
in the past few years. The approach to the problem has generally 
taken two paths, the first being a direct derivation of optimum 
filters by using the criterion of minimum mean-squared error 1 
and the other, the derivation of the so-called distortionless type 
of filter which eliminates the signal distortion component of 
error completely 2 * 3 . This approach simplifies the mathematics 
and the design problem but frequently at the expense of a greater 
overall mean squared error at the output. 

This paper extends dual input filter theory to the case of 
limiting an input into fixed elements in the system to avoid 


saturation. The introduction of this constraint is handled 
mathematically by use of the Lagrange multiplier technique. 
The analysis considers an arbitrary fixed plant dynamics in order 
that it may be applied to the practical problem of control system 
design as well as the usual filtering and estimating problem. 

The technique of combining ‘redundant’ measurements in 
order to improve the quality of estimation or control has not 
been used as widely as it could be in industry to date. This 
method will receive greater emphasis in the design of guidance 
and control systems for future missiles and space vehicles 
because component and other equipment weights are constantly 
being improved while accuracy and reliability are receiving 
greater emphasis. The appropriate use of redundance is a 
technique which can do much to improve the accuracy and 
reliability of a system. Examples of future systems which could 
use independent measurements of related quantities are: 

(1) Satellite rendezvous terminal guidance system using 
independent measurements of range and range rate. 

(2) Ballistic missile or space booster guidance using a doppler- 
aided inertial guidance system (independent measurements of 
velocity). 

(3) Satellite attitude control system which would make use 
of independent measurements of attitude, such as horizon 
sensors, star trackers, sun sensors, magnetometers, gyros, etc. 

These are only a few of the possible navigation and control 
applications of this technique. Space systems have been empha¬ 
sized because of their extreme requirements for accuracy and 
reliability. Many other possible applications exist, however, 
where these techniques could be used profitably. 

It is believed that this paper, in extending the work on the 
problem of design using multiple independent measurements 
to the very practical case of limiting to avoid saturation, increases 
the usefulness of the theory. Such a constraint on the mean 
squared value of a variable within the system is a way of restrict¬ 
ing the activity of a key intermediate variable and hence, may be 
important in designs for space as a technique to conserve power 
or energy which are premium quantities in that environment. 

Description of the Problem 

The general single-input problem of filter design with 
limiting to avoid saturation is discussed thoroughly in Chapter 7 
of Ref. 4. This desire to avoid non-linear regions of operation 
in fixed elements in a system is an important constraint on con¬ 
trol system designers. Another motivation for limiting the mean 
squared activity of a variable would be to limit the power con¬ 
sumed or other ‘costs’ which occur in the act of controlling a 
dynamical system. This paper extends the single-input theory 
with non-linear elements to dual input systems. The linear dual 
input theory is thus extended to non-linear systems through the 
device of constraining the mean squared activity of an inter¬ 
mediate variable. 
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The system to be considered is shown in Figure 1. The signals 
are stochastic variables and the constraint on the input to the 
fixed elements of the system, x (t), will involve limiting the 
R.M.S. amplitude of the saturating signal, J 2 (?) to some 
specified fraction of the linear range of the saturating element G f . 

The problem of control system design with specified criteria 
is of considerable consequence 5 . The problem of minimizing 
the mean squared error is considered subject to the constraint 

x 2 ( 0:2 of ( 1 ) 

In order to select the form of compensation, W x and W 2 , 
that will minimize the mean squared error and meet the con¬ 
straining relation, a functional may be formed by means of the 
Lagrange multiplier technique 4 ’ 5 . This functional is 

F = e 2 (t)+lx 2 (t) (2) 

where : 2 is the Lagrangian multiplier. Since e 2 (?) and x 2 (?) may 
be written in terms of functionals of the unknowns W x (co) and 
ITg 0 ^)> application of the calculus of variations to the minimiza¬ 
tion of F yields a set of two simultaneous equations just as in the 
case with the linear problem 1 ’ 2 . These simultaneous equations 


l n,(t) 



Figure 1. Block diagram of semi-free configuration dual input system 
with saturating variable x (t) 


may be solved for W 1 and W, in terms of the angular frequency co 
and the Lagrange multiplier 2. The solution for W x {co, A) and 
W 2 (co, 2 ) may be substituted into the expression for x 2 (?) and 2 
adjusted so that the constraint eqn ( 1 ) is satisfied. The value 
of 2 thus found may be used to find the appropriate com¬ 
pensation JV 1 (co) and W 2 (co), and the value of e 2 (?). These steps 
will now be developed in detail. 

Examine Figure 1, the transformed equations governing the 
dynamical behaviour of the system may be written by inspection, 
where the symbol co is the transform complex variable. 

X (co) = [S'(co) + N t (co)] W l (co) + [S(co) + N 2 (co)] W 2 (co) (3) 


Z(co)=X(co)G f (co) ( 4 ) 

E(co)=Z(co)-S(co) (5) 

The error may be written 

E (co) = G f (co) {[S(co) + N t (co)] W x (co) 

+ [S (co) + N 2 (co)] W 2 (co)} - S (co) (6) 

Making the following substitutions: 

G f (co) W t (co) = 7-t (co) ( 7 ) 

Gf (co) W 2 (co) = Y 2 (co) (8) 

Eqn ( 6 ) becomes (dropping the functional notation) 

E=Y 1 N 1 + Y 2 N 2 + lY 1 +Y 2 -l]S (9) 


Eqn (9) is in a familiar form occurring in linear filtering and 
prediction. It leads directly to the spectral density of the error 6 . 
If s’ ( t ), rt x (t) and n 2 (0 are independent stationary random time 
series the spectral density of the error may be obtained in terms 
of Y l9 Y 2 , their complex conjugates denoted by an A*; and the 
power spectral densities of the noise and signal quantities 
denoted by &w l9 and 0s, respectively. 

^ = |yi| 2 (f s + ^) + |F 2 | 2 (^+^ 2 ) 

+<P S (Y 1 Y*+ y?y 2 )-*s(y?+ y *+ y 1 + y 2 )+f s (I0) 

The mean squared value of the error is given by 

0 E (co)d co (11) 

Thus, by substituting eqn ( 10 ) into eqn ( 11 ) the following may 
be obtained 



2n 


/* 
1 2 


{L*s + <P Nl ]Y l Y l * + [cI> s + 0 N2 ]Y 2 Y : 

4 - 0 S y 2 + Y* > 2 ] — 0 s [y* + y 2 * + Y x + y 2 ] + 0 S J dco 

Similarly, 
and 

r w [ w i w * +* Sl y + w 2 w* (<p s + d> N2 ) 


X=W 1 (S+N l )+W 2 (S+N 2 ) 


( 12 ) 

(13) 




+ ( W? W 2 + W, W 2 )] d co (14) 

Now the functional Fmay be formed by substitution of eqns ( 12 ) 
and (14) into eqn (2). 


F 



+ 'F Ni )\G f \ 2 WyW*- + (<P S + 0 N2 )\G f \ 2 W 2 W? 


+ ^s\Gf\ 2 (W 1 W* + W* W 2 ) -<P S G*( W* + W *) 


- $sGf (W t + W 2 ) + 0 S ] d co + ~ 

2 71 



[(<£ s +<£ N| ) w x w* 


+ (<P S + 0 ;V2 ) W 2 W* + 0 s (W*W 2 +W 1 W*)] d co (15) 

The calculus of variations may be used with this functional 
m order to find a minimum. Following the usual procedures of 
the calculus of variations to minimize F, allow the variation 
in W 1 (co) to be e % (co) and the variation in W 2 (co) to be e % (co) 
where e is a parameter not depending upon co and r h , r h are 
arbitrary functions of co. At the same time F is subject to a 
variation d F and becomes F + d F. A necessary condition for 
the quantity F to have an extremum is 


^[T + <5F ]^ 0 = 0 (16) 

for arbitrary % (co) and rj 2 (co). 

Replacing W x (co) and W 2 (co) with their respective variations 
and carrying out the operation indicated in eqn (16) it is found 

w i \G f \ 2 (0 S + 0 N ^)+W 2 \G f \ 2 0 S -G*0 S 

+ZW 1 (0 s +0 Ni )+AW 2 0 s = Q l (17) 
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W 2 \G f \ 2 (4> s + <f>, v J +W 2 \G f \ 2 <P s -G*<P s 

+ lW 2 (* s +$ lli ) + XW 1 * s = Q t (18) 

The formal steps are given in detail elsewhere 5 ’ 7 . The result 
obtained depends upon the fact that the resulting integrals must 
be equal to zero if a minimum is to be found; and in order for 
this to be the case, each integral must contain products of func¬ 
tions whose singularities are in the same half plane. Thus, the 
quantities on the left-hand side of eqns (17) and (18) must contain 
no upper half plane poles. It is observed, therefore, that <2i (o>) 
and Q% (to) are functions with lower half plane poles only. 

Eqns (17) and (18) may be written 

(<P s + ^ Nl )W 1 (X + \G f \ 2 ) + ^ s W 2 (A + \G f \ 2 ) = Q 1 + G*<P s (l9) 
($s + <P Nl ) W 2 (X + \Gj\ 2 ) + $ s W l (X+\G f \ 2 ) = Q 2 + G*$ s (20) 

To proceed, spectral factorization 4 must be applied to the factor 
(X + | G f | 2 ). (In actual practice this is not always easy to do, it 
is useful to note that common fixed plants which are readily 
subject to this factorization are of the form K/S n or a simple 


lag.) 

(A+|G / | 2 ) = a + a“ 

(21) 

Thus 

+ + Qi Gf<P s 

(fs + <P Nl ) W,oc + * s W 2 a =—+ -4—= 

(22) 


a a 



+ + Q 2 G*r&s 

(4>s + ^)^ 2 oc + +<P s W 1 oi + ==±+-^ 

(23) 


a a 


These equations now reduce to the following simple form: 


corresponds to conservative estimates obtained from experimen¬ 
tation or theory. One can proceed by examining the so-called 
‘degenerate cases’ of Stewart and Parks 2 . 

Case I 

First, consider the case where 

^ 2 = K 2 4> Nl (31) 

Substituting this into eqn (25) 

($ s + K 2 $ Nl )R 2 + <I> s R 1 =Q' 2 + (t> 0 (32) 

Subtracting eqn (28) from (20) 

R^ n -R 2 K 2 0 n =Q',-Q' 2 (33) 

and so 

^ 1 (J? 1 -X 2 i? 2 ) = e 3 (34) 

where Q 3 is Q x ' — Q% and has poles only in the lower half plane. 

If ( 35 ) 

then 

tf> 1 + (i? 1 -K 2 R 2 ) = % (36) 

<P 1 

Observe that the left side of eqn (36) can have poles only in the 
upper half plane and the right side can have poles only in the 
lower half plane, hence 

<^ 1 + (K 1 -K 2 l? 2 ) = 0 (37) 

R, = K 2 R 2 (38) 

Substitution of eqn (38) into eqn (24) obtains 


(0 s 4- @ n t ) R\ 4- Qi 4" 

(24) 


(<*>s+%f) K 2 R 2 +$ s Ri=Q'l+<f>0 

(39) 

(&s 4“ ( Pn 2 ) 4“ = 2 2 4" 0o 

(25) 



where 


and factoring, 


Qi-Gi/fl" 

(26) 


2 (1 + K 2 ) + *2*^ —Ql+to 

(40) 

Q2~Qll^ 

(27) 

Let 

R' 2 =(1 + K 2 )R 2 

(41 

and = 

(28) 


®N Z — 


R 2 = W 2 ot* 

(29) 


(42) 


then 


, G*<P S .... RWs + *n>Qi + <I>o (43) 

<Po = —— ( ju l Next form 

“ <P s + $' Nz = $ + <f>- (44) 


Degenerate Solutions 

To apply the method of undefined coefficients at this point 
as was done by Wescott 1 would be extremely laborious, if not 
impossible, as the Lagrange multiplier is also unknown. 
In many practical applications of this theory one may be perfectly 
free to assume similarity of bandwidth either for signal and noise 
or for the two noise inputs. By selecting the similar power 
spectral densities to be identical in bandwidth a more conserv¬ 
ative approach may be made and the problem becomes tractable 
once again. This technique is common in engineering practice 
where much of the quantitative data used in a design problem 


thus 

(45) 

<p cf> 


Observe that the right side of eqn (45) contains upper half plane 
poles only and thus this equation leads directly to the Wiener- 
Kolmogoroff solution for the optimum filter 5 . Hence, after 
substituting for (j} 0 the relation is obtained 


1 ~ G*(c o)<E s (<p) 

4> + (to) La - (®, A )<t>~ (ffl)J+ 


( 46 ) 
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The bracket symbol [ ]+ is taken to mean that a partial fraction 
expansion is formed and the terms with lower half plane roots 
are discarded. The filters which minimize e 2 subject to the con - 

straint that x 2 < <r s 2 may be found as a function of the Lagrange 
multiplier by substituting eqn (29) into eqn (46), 


Next, define some new functions 

«i= 


l + c : 


-Ri 


W 2 (co,X) = - 


1 

1 

" <?/(©)** (to) 1 

1 +K 2 

_« + (co,X)4> + (co)_ 

( co , X)4>~ (co)_ 


,, 1+c 2 

~2 $N t 


(47) 


(48) 


From eqns (28), (29) and (38) it is found that 

W t oc + =K 2 W 2 a + 

or 

W 1 = K 2 W 2 (49) 

Next, substitute these filter functions into eqn (14) and find the 
value of 2 which satisfies the constraint on x 2 . 

Case II 

The next problem is to derive the solution for the case where 
it may be assumed 

- 2 % 


$n 2 = c 2 


With this substitution, eqns (24) and (25) become 

(<p s +<p n j r 2 + 0 s r 2 =q[+4i 0 
+($S + C 2 <P S )R2 = Q2 + ct>0 

Factoring eqn (52) 

[i^i + R 2 (1 + C 2 )] = 22 + ^0 

and letting 

^ = 0 + 0‘ 


find 


^ + C« 1 +R 2 (1+c 2 )]=^- + -^ 

0 (/) 


(50) 

(51) 

(52) 

(53) 


(54) 


Substituting eqns (59) and (60) into eqn (58) find 

($S+^tf,) = 0i + 0i 


Let 

then 


4>iR[=~z: + 


Q\ i 0o ^s/7 


(59) 

(60) 

(61) 

(62) 

(63) 


0i 0 i 0i 

The first term on the right-hand side of eqn (63) contains only 
lower half plane poles. The remainder of the equation may be 
made compatible with the left-hand side to obtain 


1 


R 'i = 

1 i + 




0 + L-0 1 <f>\ J 


(64) 


Applying eqns (28) and (59) the final solution for J+, (o>, X) 
may be obtained 

W l {m,X)= 


i 

i — 

<N 

+ 1 
^ H j 

1 

a + (co, X) 

L c 2 J 

U»J 


0o (to. A) $s(c0)P(a>,X) 

L 4>I((0,X) 4>i(co,X) . 

Combining eqns (29) and (55) find 


W 2 tx + =p- 


l + c 2 


and hence 


w > 


x + 1 + C 2 


(65) 

( 66 ) 

(67) 


The left-hand side of eqn (54) has poles in the upper half plane 
only. The first term of the right-hand side has poles in the lower 
half plane only, hence as before it must be dropped from the 
equation. From eqn (54) is obtained 


R 2 = 





(55) 


Although this equation is somewhat complicated, the only 
unknown quantities now appearing are R 2 and R 2 , the unknown 
Q 2 having been eliminated. For notational convenience define 


P(cd,A) = 


1 


1 


00 


^+c70 + L0 

Now, substitute eqns (55) and (56) into eqn (51) and obtain 


(56) 


(<p s +<P Wl )« 1 +<P s ^- I |i I j=Gi+^ 0 

Factoring, obtain 


(57) 


* 7-7 () + $N t Ri=Q’i+(p 0 - $ S P 


(58) 


As before, these filter functions are substituted into eqn (14) 
and the value of X which satisfies the constraint on x 2 is found. 
This value of X then enables the evaluation of W x and W 2 
and the <? 2 . 

Case III: The Distortionless Filter 

The techniques for the design of the distortionless filter with 
a saturation constraint will now be developed. This type of 
filter is somewhat simpler to analyse and design than the Wiener 
filter. There are many practical applications for this type of 
filter, particularly if the signal power spectral density is not 
known or if the signal is deterministic rather than stochastic 
in nature. 

Examining Figure 1, it is observed that the error is given 
by eqn (9) where the symbols are defined as before. Eqn (9) is 
repeated here for convenience, 

E=Y 1 N 1 + Y 2 N 2 + [Y 1 + Y 2 -1]S (9) 

In order for this filter to be ‘distortionless’ the last term on the 
right-hand side of eqn (9) must be zero, hence 

Y 1 + Y 2 = l (68) 
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and now the error is due to noise terms only, 

E=Y 1 N i +(l-Y l )N 2 (69) 

Eqn (69) permits the finding of the power density of the error. 
$ E (©) = | Y x (ft))| 2 <P Nl (co) + 11 - Yi (co)| 2 <P N2 (cd) (70) 

The functional eqn (2) remains the same with the form of e 2 
changing to that of a distortionless filter. Thus, 

1 r °° y r °° 

F = —— # £ (o>)dco+ — ^ x (<w)dco (71) 

2 71 % — oo 2 7t J _ qq 

For this case then, the functional to which the calculus of varia¬ 
tions must be applied may be written by substitution into eqn (71). 


F = 


_1_ 

2tc 


% oo 

(l^l^ + l^l 2 ^)^ 

- 00 



1 00 

[1^11 2 + \Wl\ 2 (#S+^N 2 ) 

— 00 


+ Wf W 2 <P S + W x W?<p s \ dco 


(72) 


Recalling that Y x = G, W x and Y 2 = G f W 2 apply the calculus 
of variations to find an extremum of eqn (72). This results in 
deriving the following two simultaneous equations: 

W x \ Gf \ 2 + ($ 5 + < &N l ) Jr ^W 2 <P s = ~~Qi 

W 2 | C 7 y | 2 @1^2 + 2 W 2 ($£ + < &N 2 )^~ — ~~ 62 

Eqn (68) may be written 

w 1 + w 2 =~ 

°7 

After algebraic manipulation and the use of eqn (75) obtain 

<P Nl W 1 (X + \G f \ 2 ) + <P s lG} 1 = -Q l (76) 

<S> N2 W 2 (A + \G f \ 2 ) + F S XGJ 1 = -Q 2 (77) 

Subtraction of eqn (77) from (76) obtains 


(73) 

(74) 

(75) 


(X+\G f \ 2 )(<P Nl W x -<P Nl W 2 )=Q 2 -Q x (78) 


Define some new functions 



4y N =(X + \G f \ 2 ) < P Nl . 

(79) 



(80) 


* 0,3 =22“6l 

(81) 

Eqn (78) now takes the form 



^' Nl W l -cP / N 2 W 2 = Q 3 

(82) 

Using eqn 

(75) find 




(83) 

Hence 



(84) 

Letting 



(85) 

it is found 


w x =—+— — 

<t> $ 

(86) 


Notice that the situation again exists where the left side of 
eqn (86) contains only upper half plane poles and only a portion 
of the second term on the right may be retained. The portion 
of the second term is found by applying the methods of Wiener 
and Kolmogoroff and the resulting optimum filter transfer 


function is 


Also 


W^coJ l) = 


_1_ 

<t> + 


- <T - 


+ 


W 2 (co,X)=^-W 1 (a,X) 


(87) 

( 88 ) 


These transfer functions are substituted into eqn (14). Then the 
constraint relation eqn (1) is applied by selecting that value of A 
which satisfies the constraint. This value of A may now be used 
to evaluate W-± and W 2 and then eqns (70) and (11) provide the 
value of e*. Notice that if this value of the mean squared error 
is too great for a reasonable system design, a compromise 
constraint value may be chosen. In fact, in a practical design 
problem one would undoubtedly examine the relation between 
the value of constraint on power (or saturation) as compared 
with the mean squared error resulting from such a constraint. 
The following numerical example should clarify the above 


concepts. 


Numerical Example 

Examine Figure 2 which represents the example problem. 
Select G f = 1-0 (no dynamic portions). This will reduce the 
algebraic manipulations and thus aid in clarifying the work. 
For this example assume 


<p Sl (to) = 


4 

co 2 + l 


$ s (w) 


co 2 (co 2 + l) 

^ (a))= ^+I 

Following the techniques outlined in Case III (the distortionless 
filter) evaluate eqns (79), (80). 




*,-(/• + !) 


<Z^=(A + 1) 

and eqn (86) indicates 

<k + 4> =(<p' Ni +<p' Ni ) 

= (X+i) 


CD 2 (CD 2 + 1)_ 
2 

co 2 +1 


1 J r\j2j(X> 

l-JljCD 

_j(D(l+jC0)_ 

_-ja>(l-jco)_ 


Following eqn (87) find 




r 2 (A +1 )(-j©) 

- <S *>- - 

+ 

_(1 +jco)(l—y/2jco). 


(89) 

(90) 


(91) 


( 92 ) 
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and expanding eqn (92) obtain 

1 __ Kj t K 2 
(j>~ l+j® 1—^/2 jco 

then 

X 1 = ^^=C 0 (A) 

1 +\/2 

Now evaluating eqn (87) 


in order for a solution to be possible at all. This corresponds to 
the lowest value of constraint which can be met. This solution 
corresponds to the following filter functions, 


»i (*>) = 

W 2 (co) = 


Wi (co, X) = C 0 (X) 

and from eqn (88) 

W 2 (co,A) = 1~ C - o( f Ja = 
1 + ^2 jco 

From eqn (1) 


jco 

1 X-\j2jco. 


Cp (1) jco 1 +( v / 2 — C 0 )jco 
1 + -J 2 jco 1 + *J2 jco 


-0453 jco 
1+^2 jco 

1 + 1-867 jco 
1 +yj2 jCO 


x 2 (t)<c 2 > 


i r +j '°° 


$x(s)ds 


Jl= 2nj] l^il^s+^ds 

J 2 =^r\ +1X \W 2 \ 2 ($ s + <P N2 )ds (99) 

Z7Z j J -j 00 

i f+;» 

J 3 = ~- 

27c jJ -JCO J 

Substituting for and JF 2 from eqns (95) and (96) the integral 
is evaluated to obtain 


This solution would be mechanized by Bendat’s method 3 as 
shown in Figure 3. By the more direct method (after Figure 2) 
it would take the form of Figure 4. In these figures p is meant to 
represent d/dz 1 and replaces jco. Eliminating the integrator and 
recognizing the fact that in Figure 2 an integration was assumed 
in the channel containing W 1 (p), for mathematical convenience, 
it is possible to show another mechanization for the filter which 
is that of Figure 4. 


where s is the Laplace complex variable. Next, write eqn (97) 
with reference to eqn (14), 

_ if +J * cp 

X 2 CO =— {W t (s) W.i-s) j> s (s) + <P Nl (s)] 

J — j00 

+ W 2 (s)W 2 (-s)[<P s (s) + <P, 1 (s)-] 

+ (s) W 2 (-s) <P S (5) + W 1 (-s)W 2 (s) <P S (s)} ds (98) 

Breaking the integral into three parts, 

. i r + -. .... i 



Figure 2. Block diagram of example problem 


s 2 (t)+n 2 (t) 1+1-867p 

if wgp 


Tj s 2 (0 


Filter j 

s,(0+n;(t) |~1 .L*‘ s 2 (t ) 

-* 1/p --—*• 

Integrator 

Figure 3. A possible mechanization of the example 


s.ttj+nid) 


s 2 (t) 



Figure 4. Another mechanization of the example 


_ 3 Cq (A) 6V2C 0 (A) „ 

l A tic/: ‘ a scr ' 


9-656 9-656 

_V2C 2 (A) 


Summing these and applying eqn (97), C 0 (7) must satisfy the 
following relation 

0-957 C 2 0 + 0-877 C 0 +1-25 <a 2 (101) 

From eqn (101) it is obvious that [see eqn (96) and invoke 
physical realizability] 


Therefore, 


4 (0-957) (1 -25 - a 2 ) < (0-877) 2 


<>1-05 


Next, examine how the constraint has affected the mean 
squared error. For this filter write e 2 from eqns (70) and (11) 

I M (5)| 2 * Nl <s) + 11- W x (s)| 2 4> Wl (s)} ds 
“ JJ ~ JC0 (106) 

and evaluating this integral we obtain 

?=l-48 (107) 

The minimum mean squared error may be found from eqn (72) 

with X equal to zero. The minimization procedure for the distor¬ 

tionless filter, when it is not constrained to avoid saturation, is 
straightforward 7 and is therefore not shown here. The minimum 
mean squared error for the unconstrained case is 

?| min =0-670 (108) 

This is a value less than half of that given in eqn (107). 
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The activity to be expected of the constrained variable x 
may be found for the unconstrained case by using the fact that 

i f + /°° 

P=7 + -L <D s (s)ds (109) 

2 ft / J - j 00 

Evaluating the integral and adding eqn (108) 

7 = 2-67 (HO) 

Thus the bound on the constraint cr s 2 is such that all values of 
a* greater than 2-67 may be satisfied by the ‘optimum 5 distortion¬ 
less filter. 

The author gratefully acknowledges the helpful advice on this 
problem received from Professor C. T. Leondes, Department of 
Engineering, University of California, Eos Angeles , California. 
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DISCUSSION 


D. Siljak, Electrotechnical Faculty, University of Belgrade, Macvan - 
ska 8, Belgrade, Yugoslavia 

This paper is a useful contribution to the literature in that it extends 
the optimization of mean squared error with saturation constraint 
to the design of dual-input filters. The proposed design procedure can 
be applied to automatic control systems with multiple independent 
measurements which have certain important advantages. However, 
in the application of the procedure to the design of control systems, 
problems may arise due to the fact that the stability aspect of the 
control problem is neglected. To hold the peak value of a signal 
within the linear range by controlling its R.M.S. value may be inade¬ 
quate and the control system would exhibit poorly damped transient 
modes. Moreover, it is often difficult to determine the appropriate value 
of the R.M.S. constraint; even the controlling of the maximum value 
of the R.M.S. signal effectively limits the tendency of the signal to 
exceed its linear range. 



Figure A 


By application of the generalized Mitrovic method", the problem 
of minimizing the mean squared error may be considered as subject 
to the constraint that the degree of relative stability is better, or at least 
equal to the prescribed value 3 . The minimization procedure will be 
shown briefly by the following example. 

Consider the control system shown in Figure A, with the specifi¬ 
cations 

Gf ^ = s (0-001 s 2 +0-025 s +0-25) 


K(s + XS). f . 


Is 

71S 2 


0 ) 


u ( t ) = unit step function ; <fi N (s) = y N lit 

It is required to determine the parameters K, 2 and d of the integral 
compensator W(s) so as to increase the system velocity constant and 
minimize the mean squared error while maintaining the overshoot 
of the step-function response below 30 per cent of its steady-state 
value. At first the noise component is not considered. 


As known, the choice of the parameter <5(<^ 1) is not critical and 
the value 0*04 may be accepted. Then, if the numerical value of ys is 
2 n, the power density of error (j>is(s) corresponding to the signal s(t) 
is given as 

0-002 s 3 4- 0-05 s 2 + 0-5 s-1-0-02 _ 

^ = 0-001 s 4 4- 0-025 s 3 + 0-25 s 2 + (K+ 0-01) s +0-04 KX 

(2) 

The denominator of <!>e(s) is the characteristic polynomial of the 
system under investigation. By substituting K + 0*01 = £ and 
0-04 K = rj in eqn (2), one may plot in the usual fashion 2 " 4 the charac¬ 
teristic curves Ft, as shown in Figure B. These curves determine in the 
figr) plane the relative damping region which corresponds to 
certain value of the relative damping coefficient £. So, for f = 0-4 the 
relative damping region is determined by the curve -T0-4 and shown 
shaded in Figure B. For the values of £ and rj lying in this region all 
roots of the corresponding characteristic equation will have the 
relative damping coefficient greater than 0-4. 

Applying the above substitution and using the well-known 
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technique 1 , one may evaluate the mean squared error 7 from eqn (2) 
as a function of £ and r\ 


/(&*)= 


0-025 1] 2 -0-375 & -1-55/7 + 10~ 4 <f —0-6510' 
0-625 F+FF &25&I 


For different values of 7, a family of the curves determined by the 
eqn (3) is plotted in Figure B. 

Now, in order to minimize the mean squared error and simulta¬ 
neously maintain the prescribed degree of relative stability, the solution 
of the control problem under investigation is found as a constrained 
minimum of 7, which is located at the point M x of the diagram of 
Figure B. Within the relative damping region of £ = 0*4 the point M l 
has minimum value of the mean squared error. 

The curve F 0 . 4 and the tangents T x and T 2 drawn from the point 
enable the roots of the characteristic equation corresponding to 
that point to be evaluated without any calculation 2-4 

*1,2 = — 4-1 ±j 9*39; ^ = —0-864; ^ = — 16-4 

The smaller real root s 3 and the zero of the corresponding closed-loop 
transfer function form a dipole whose effect may be neglected. The 
other real root is large and its effect is also negligible. The step-function 
response will therefore be governed only by the pair of complex roots 
*1,2 whose value of damping coefficient £ = 0-4 ensures that the 
overshoot is less than 30 per cent. 

From the coordinates of the point M x {^ ~ K - f- 0-01 — 1*89; 
;y = 0*04; KX — 1*52) the values of the compensator parameters are: 
K— 1*88 and X = 20*2. The system velocity constant is 38 times 
greater than the velocity constant of the uncompensated system. The 
constrained minimum of the mean squared error is 7 — 0*307. It is 
interesting to note from Figure B that the absolute minimum 7 — 0*187, 
which is located at the point M, falls outside the relative damping 
region corresponding to f = 0*2 and therefore results in a poorly 
damped system. 

In a similar manner 3 , the component of the mean squared error 
corresponding to the noise can also be expressed as a function of £ 
and p, which for = 2 n has the following form 


J ($,*!) = 


— 6-25 >7 — 25 1 2 
0-625 >7 + £ 2 — 6-25 £ 


(4) 


Using this equation, the same reasoning outlined above may be 
applied to the noise case. Furthermore, by the proposed techniques, 
it is also possible to introduce the quadratic corstraint of the satura¬ 
tion signal into the optimization in the d>£/y plane 3 . 
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R. S. Gaylord, in reply 

I thank Professor Siljak for his very interesting discussion of my paper. 
It is particularly interesting to see how he has applied this theory to 
closed-loop control system design. Although the work that he describes 
is not directly concerned with dual-input systems, nor is the problem 
of added constraint on x ( t ) treated, it is clear that his topological 
methods can be applied to these considerations by mapping the 
constraints into his £, rj plane. 

An additional important observation is that Professor Siljak must 
choose a form for his compensation network [see eqn (1) of the dis¬ 
cussion], whereas with the approach given 1 this is not necessary. 
However, if the method fails to find a solution due to this constraint, 
then another network may be chosen, and so forth. The network found 
by such a procedure may not be ‘optimum’ in the sense of the paper, 
and further, the introduction of the distortionless constraint into 
Professor Siljak’s method would be very difficult, that is, to satisfy 
the equation 

W t G f +W 2 G f = 1 

may narrow the region in the £, ?) plane to the point where the solution 
may not exist. 

As implied by Professor Siljak’s opening remarks, os 2 is not a very 
good way to constrain the signal amplitude from saturating. It seems 
that a more suitable application is to conserve fuel or energy, then the 
practical choice is much easier. Of course, os 2 need not be chosen in 
the problem, as the trade between .x 2 and e 2 may be plotted and 
compared visually. 

Reference 

1 Newton, G. C., Gould, L. A. and Kaiser, J. F. Analytical Design 
of Linear Feedback Controls. 1957. New York; Wiley 


F. Csaki, Department of Automation, Polytechnic University, Egri 
Iv 18, v , Budapest XI, Hungary 

In his very interesting paper the author has considered a dual-input 
system with one saturation constraint. However, this problem is only 
a special case of the multi-variable systems with many constraints. 
The problem can be generalized as seen in Figure A. 
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Here all lower case letters denote time functions, for example 
s. k , n.jc , r.fc, m.j, c.i, i-u e.u and x.i are row vectors, while wf cj , 
and MV# are the weighting-function matrices. The task is to minimize 
the following function: 

f 2 (0 = tr {e,_ e , + A x L *,} = tr {4> e ,. e , (0) + X<p Xl * x , (0)} 

for stochastic processes by an optimum matrix of the cascade con¬ 
troller wffi, where tr denotes the trace of a matrix, i.e. the sum of its 
diagonal elements. 

Instead of correlation functions we can use the power spectral 
densities: 

f 2 (0 = ^7 I tr{cl) ei . ei (s)+X(t> xl ' Xl (s)}ds 

2njJ j co 

where s = jco . The condition of constraints can be expressed as 

J-. ti'{(l> xl ' X ,(s)}ds^a 2 

2 71/ J - jco 

After some mathematical manipulations the following expression 
can be obtained for the matrix integrand 

=M)-K k (s)K-(s)wf t (s) 

-W l f r (-s)W J c , k ,(-s)$ rk . h .(s) 

+ w(.j. (- i) WJ. L . (- 5) K'U (s) K (S) wf t (s) 

+ A W? r (-s)W^(-s) 4> r ,. rk (5) (s) H7,(s) 

Let us now introduce an auxiliary power spectral density matrix 
cf) au H (s) by equating the last two terms on the right-hand side with 
the following expression. 

W(,y (- s) w rk . (- s) (t> 0k - llk (s, X) W k c j (s) Wft (s) 


Without going into details, the optimizing transfer function matrix 
of the controller can be expressed in the following explicit matrix 
relation 

W&is, k)=[fa ak (s. A )]" 1 {LK^Xy }- 1 Wfa-s) 

x [(Wtf (s) w^i-syy 1 } + [(VF/(.s) w/j.f-sjf r 1 

The solution for (s, X) may now be substituted into the expres¬ 
sion 

and the Langrangean variable 2 can be adjusted so that the condition 
of constraint will be satisfied. 

These results are the generalizations of some results obtained in 
the paper under discussion, the details will be published soon in 
Periodica Polytechnica , Budapest. My results obtained by the so-called 
simplified method are generalizations of the results obtained by 
Wiener, Amara, Leondes, Newton and others. 

R. S. Gaylord, in reply 

I appreciate the important addition made by Professor Csaki in 
generalizing the problem of multivariable systems with constraints. 
Due to this valuable addition to the theory one can now consider many 
signals, controllers, plants and constraints. It is true, however, that 
the amount of algebraic manipulation greatly increases with the 
amount of consideration desired. Also, the plant must still be of a very 
simple form and the signal and noise quantities known in advance, 
at least as a power spectral density. 

The so-called distortionless method eliminates the need to have 
a known random process as a signal quantity; but it does not appear 
to be so simple to include this kind of constraint in the more general 
formulation given. 


Signalling and Prediction of Failures in Discrete 
Control Devices with Structural Redundancy 

M. A. GAVRILOV 


Summary 1 


This report examines the problem of equivalent conversions of the unit 
structures of discrete control devices whose functioning algorithms are 
described in the language of conversion tables. An algorithm is de¬ 
scribed which may be used to convert from any complex structure of a 
system of units, including those with feedback which require for each 
conversion an expenditure of nr units of time (n is the number of units 
acting in sequence upon each other), to one equivalent unit which 
requires r units of time to perform the same functioning algorithm. 
An evaluation is made of the change in structural complexity of such 
equivalent units. Methods of signalling and predicting failures in dis¬ 
crete control devices with structural redundancy are considered. 

Sommaire 

Ce rapport traite des conversions equivalentes de structures unitaires 
d’elements de commande discontinue, dont les algorithmes opera- 
tionnels sont decrits par des tables de conversion. On montre un 
algorithme pouvant etre utilise pour convertir, n’importe quelle struc¬ 
ture complexe representant un ensemble d’unites, y compris des unites 
a asservissement qui demandent pour chaque conversion nr unites de 
temps (/? est le nombre d’unites agissant en sequence les unes sur les 
autres), a une structure unitaire equivalente qui demande r unites de 
temps pour effectuer le meme algorithme operationnel. On evalue le 
changement de complexity structural de telles unites equivalentes. On 
examine des methodes pour signaler et prevenir les pannes de fonction- 
nement des elements de commande discontinue avec redondance 
structural. 

Zusammenfassung 

Dieser Beitrag untersucht das Problem der gleichwertigen Umwand- 
lung der Elementarstrukturen unstetiger Regenerate, deren Funktions- 
schema mit Hilfe von Umwandlungstabellen beschrieben wird. Es wird 
ein Verfahren beschrieben, mit dem man jede komplexe Elementar- 
struktur, darunter auch solche mit Ruckfiihrung, die fur jede Um- 
wandlung nr Zeiteinheiten brauchen, in ein gleichwertiges Schema 
umwandeln kann, das r Zeiteinheiten benotigt, um die gleiche Funk- 
tionsvorschrift zu erfullen (dabei ist n die Anzahl der Einheiten, die im 
Wirkungsablauf aufeinander folgen). Die Anderungen in der struk- 
turellen Komplexitat derartiger gleichwertiger Elemente werden ge- 
schatzt. Es werden Verfahren fur Anzeige und Vorhersage von Aus- 
fallen in diskreten Regelgeraten mit redundanter Struktur betrachtet. 


In solving problems of providing reliable operation of automatic 
control devices, a great deal of attention is devoted to the use of 
methods involving the application of structural redundancy. 
These include all possible methods of duplicating individual 
elements within units, as well as the more common methods of 
providing redundancy of all the necessary elements and units on 
Uie whole with the least possible number of additional elements. 
The ever-increasing practical use of methods of structural 
redundancy is a result of the fact that, in present complex 


automatic systems, the control devices require such a large I 
number of individual elements to perform their functions that 
even though the elements may have a very high reliability, the ' 

necessary reliability demanded of the entire device cannot j 

be achieved. \ 

A number of works 1 " 6 is devoted to the question of the 1 
introduction of structural redundancy and the determination \ 
of the minimum number of additional elements necessary to I 

achieve the prescribed reliability of the device on the whole. 

For discrete control devices it is most natural and suitable to S 

examine the required value of operating reliability of the device 
as being prescribed by a certain number of elements which ! 

simultaneously fail during operation while nevertheless per¬ 
mitting the device to perform accurately the control algorithm I 

assigned to it 7 . 

The author of the present report showed 3 that when the l 

problem is treated in this manner, the determination of the 
minimum number of additional internal elements necessary to 
achieve a given reliability completely coincides with the task 
of determining the minimum number of additional symbols in 
the construction of correcting codes with correction of the 
corresponding number of errors. In the same article a method 
was given for constructing tables of states which provide for 
a realization of the structure of a discrete control device having 
the required reliability. 

The proposed method links the problem of constructing ! 

such a device to the distribution of the states of its internal \ 

elements along the vertices of a many-dimensional cube of i 

single transitions in such a manner that the number of transitions J 

(distance) between the vertices, selected for the corresponding 
stable states of the device, would be no less than: j 

D=2d+1 (j) j 

where d is the number of simultaneously failing elements with 
which the devices must still exactly perform their control : 

algorithm. j 

In differentiating the demands on reliability (namely, separat¬ 
ing them from the viewpoint of the number of simultaneously j 

failing elements), first, into that for which the device must 1 

accurately perform a given control algorithm and, second, into ! 

that for which it must not provide any actions at its outputs, the 1 

value of the distance between vertices selected for the stable : 

states must be no less than: j 

D=2d+A + 1 (2) I 

where A is the number of simultaneously failing elements in 
addition to d for which the indicated second condition of ; 

reliable operation of the device must be fulfilled. I 

In discrete types of devices which have reliability as a result l 

of structural redundancy, the required reliability is retained f 
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only until the moment of onset of permanent failure of even one 
of the elements. 

In fact, let the prescribed probability of failure of the entire 
device on the whole require that the given control algorithm be 
exactly performed with the simultaneous failure of d elements. 
Then, with a permanent failure of any one of the elements, the 
device will capably perform the control algorithm only upon 
the simultaneous failure of d — 1 elements; that is, it will have 
a probability of complete failure which is less than prescribed. 

Particular importance is therefore devoted to rapid signalling 
of failure of individual elements or their prediction, which 
permits one to take timely measures to replace the faulty 
elements or other measures which will return the probability 
of failure of the entire device to its prescribed value. The present 
report is devoted to an examination of the fundamental possibil¬ 
ities of providing such signalling or prediction for automatic 
control devices designed on the basis of the principles described 
by the author 3 . 

First it is shown that the table of states constructed according 
to the principles contains all the necessary information on 
failure, both generally for all elements as well as for each of 
them individually, and, even more, on the nature of the failures. 

Those states of internal elements which correspond to the 
stable states of the corresponding table of transitions and which 
are distributed, as was pointed out above, in the vertices of a 
many-dimensional cube of single transitions with a distance one 
from the other of not less than D, are called basic. To each of 
these states there must correspond a particular state of outputs 
which provides for the performance of the prescribed control 
algorithm. 

Let the number of inputs of the discrete device be equal to a 
and let it be given that, to perform the control algorithm with a 
prescribed degree of reliability, that is, in the presence of 
simultaneous failure of d internal elements, it is necessary 
to have K internal elements. Then each of the basic states may 
be characterized by a certain conjunctive member of a Boolean 
function of length a + K. In accordance with this the table 
of states contains, on the left-hand side, a + K columns of 
which a characterizes the states of the inputs and K charac¬ 
terizes the states of the internal elements. The binary number 
characterizing the state of the internal elements corresponds 
to a particular vertex of the many-dimensional cube, selected 
in distributing the given basic state.* 

The failure of any element is characterized by a change in 
the binary number, corresponding to a given basic state, from 
zero to one or one to zero. The first is called a 0 —> 1 type 
failure and the second a 1 —>■ 0 type failure. Each such failure 
transfers the basic state to an adjacent vertex of the many- 
dimensional cube. The simultaneous failure of any two internal 
elements transfers the basic state to a vertex two units removed 
from the vertex selected for the given basic state; it is adjacent 
to any vertex to which the basic state was transferred by the 
failure of any one of these two elements. 

In order to provide exact performance of the control 
algorithm upon the failure of internal elements, each of the 
states to which the basic state is transferred upon the failure of 
any number of elements within the prescribed limits (that is, 
inclusive to d) must compare in the right-hand side of the table 

* All references made below to internal elements with an identical 
base pertain to inputs and sensing elements. 


of states to the same state of outputs as the basic state. Therefore, 
for each stable state of the table of transitions, for the case of 
structural redundancy, there must correspond a particular 
combination of states consisting of the basic state and all the 
states to which it transfers upon failure of the internal elements. 
All of these states are adjacent to one another, forming a certain 
multiple of adjacent states. This multiple is called a set of basic 
states. 

First it is shown that the set of adjacent states, together with 
the basic states, may be described by a symmetrical Boolean 
function whose active numbers represent a natural series of 
numbers from K — d to K. 

Let there be any state f i0 corresponding to one of the basic 
states and let this state be characterized by a row in the table 
of states containing K x zeros and K 2 ones, where K x -F K 2 = K. 
Then, with d = 1, the collection of adjacent states contains 
all the states differing from the basic by the replacement of one 
variable by its reciprocal. More precisely, they are K , while K t 
of them corresponds to a failure of the type 0-> 1 and K 2 to 
a failure of the type 1 —> 0. It is easy to see that the sum of these 
states may be characterized by the symmetrical function: 

2 fn = S K - i(xi,x 2 , + •••> x Xi+a , 2 ) 

if the basic state is considered a symmetrical function of those 
variables with an active number equal to K, namely: 

fiO ~Sk( X 1' X 2> •••? + X K i + 2' "'’ X Ki+K 2 } 

The sum of the basic and set of adjacent states is thus 
characterized by the symmetrical Boolean function: 

fiO+^fn =Sk- l, k(. x U X 2> •• •> X K> X K+l’ X K+2> • * *? XK 1 +K 2 ) 

If d = 2, the set of adjacent states consists of all states 
differing from the basic by the replacement of one variable by 
its reciprocal, the number of which, as was pointed out, is equal 
to K = C 1 k> and two variables. The number of the latter is 
obviously equal C 2 x, and since each of them differs from the 
basic by a change having a value of two variables, their total 
S/J 2 corresponds to the symmetrical function: 

%fi2 == S K -. 2 ( X U x 2> ‘--’ X Kii x Ki + ii x Ki + 2> *”’ X Ki + K 2 ) 

The Boolean function characterizing the basic state and the 
entire set of adjacent states is thus a symmetrical function of the 
type: 

Zio + 2/ii+S/a 

= &K.-2, K-l, k( X 1> X 2f *5 X Kt> X K i + i? X Kt + 2 > ** *» X Ki +K 2 ) 

It may be proved in an analogous manner that in the general 
case, with the simultaneous failure of d internal elements, the 
basic state and the set of adjacent states may be characterized 
by a symmetrical Boolean function of the type: 

Sjc-d, K-d+l, ...,k( X 15^2? •**> X K}> X K 1+ p X Xi + 2’ * * •» X Ki +K 2 ) 

Thus, the class of reliable structures of discrete devices is, 
with respect to internal elements, a class described by symmetrical 
Boolean functions of a special type, which facilitates their 
realization since these functions have been most widely studied 
and may be economically realized with the aid of different types 
of threshold relay elements, including electromagnetic relay 
elements with several windings 8 . 
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The basic state is designated as f, and the set of adjacent 
states corresponding to it as N h assuming that -f jV f = F t . 

The table of states of a discrete control device consists on 
the left-hand side of all sets F t combined with the corresponding 
values of inputs. For each of these sets there corresponds on the 
right-hand side of the table, as was pointed out above, a state 
of outputs which provides for the performance of the control 
algorithm. One more output is added for which is included in 
the table of states a zero for each of the basic states and a one 

for any of the states which are included in the sets of adjacent 
states. 

Since the latter corresponds to the failure of any one or to 
the simultaneous failure of several internal elements, the appear¬ 
ance of a one at this output occurs only by means of a decrease 
m the reliability of operation of the discrete device and may be 
used to signal the presence of a failure. 


A —A 


Figure 1 

For example, let there be a discrete device with three inputs 
and one output {Figure 1) and an action, equal to one, must 
appear at the latter in the subsequent sequence of change of the 
states of the outputs: 

0 0 0 
1 0 0 
1 1 0 
1 1 1 
0 1 1 

Any subsequent change of inputs must lead to the appearance 
of an action at the output equal to zero, while the further appear¬ 
ance of an action at the output equal to one occurs only by the 
repetition of the indicated sequence of change of the states of 
the inputs. With any other sequence of change of the states of the 
inputs, the action at the output must remain equal to zero. 

The corresponding table of conversions is given in Table 1. 
Here it may be seen that it is necessary to provide for four stable 
states, which is possible with the aid of two internal elements. 

When it is necessary that the aforementioned discrete device 
performs exactly a preassigned control algorithm in the event 
o the simultaneous failure of one of the internal elements, five 


Table 1 


000 

100 

110 | 

010 

Oil 

111 

101 

001 

(1)° 

(1)° 

1 2 

4 

(l) 1 

4 

4 

4 

— 

4 

! (2)° 

4 

— 

3 

— 

_ 

— 

— 

4 

— 

1 

(3) 0 

4 

_ 

1 

(4) 0 

(4)“ 

(4) 0 

(4)° 

(4)° 

(4)° 

(4)° 


internal elements are required, as seen in Table 5 of reference 3. 
The following distribution for the basic states is chosen: 

0 0 0 0 0 
10 110 
0 10 11 
1110 1 

Then the table of states will have the form shown in Table 2. 
In agreement with what was mentioned above, let us add the 


output C 0 , in the column of which are written zeros in 
all the rows of the table of states corresponding to f t and ones 
ln all the rows corresponding to N t ( Table 3). Then this output 
will signal the presence of a failure of any one or several of the 
internal elements. 


Table 2 


ABC 

F 

1 

z 

0 0 0 

Fn 

0 0 0 0 0 

0 

0 0 0 

Ft 

0 0 0 0 0 

0 

0 0 1 

F l 

1110 1 

0 

0 0 1 

Ft 

1110 1 

0 

0 1 0 

F 1 

1110 1 

0 

0 1 o 

F, 

1110 1 

0 

0 1 o 

F .i 

1110 1 

0 

0 1 1 

F 1 

0 0 0 0 0 

1 

0 1 1 

f 3 

0 0 0 0 0 

•1 

0 1 1 

F& 

1110 1 

0 

1 0 0 

F 1 

0 0 0 0 0 

0 

1 0 0 

F , 

1110 1 

0 

1 0 0 

F, 

1110 1 

0 

1 0 1 

1 F i 

1110 1 

0 

1 0 1 

F 3 

1110 1 

0 

1 o 1 

F t 

1110 1 

0 

1 1 0 

F 1 

10 110 

0 

1 1 0 

F, 

10 110 

0 

1 1 0 
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1110 1 

0 
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F i 

1110 1 

0 

1 1 1 

Ft 

1110 1 

0 

1 1 1 

F't 

0 10 11 

0 

1 1 1 

f 2 

0 10 11 

0 

1 1 1 

Ft 

1110 1 

0 


In this table: 


0 0 0 0 0 

10 110 

0 10 11 

1 0 0 0 0 

0 0 110 

110 11 

0 10 0 0 

11110 

0 0 0 1 3 

0 0 1 0 0 F - 

10 0 10 F * 

0 1111 F * 

0 0 0 1 0 

10 10 0 

0 10 0 1 

0 0 0 0 1 

10 111 

0 10 10 


1 1 1 0 1 
0 110 1 
10 10 1 
110 0 1 
1 l l 1 1 
1110 0 


If one places the action from this output into a computer 
and determines the number of times that actions equal to one 
appear at this output during a certain time interval, the answers 
from the computer may be used to predict an approximation 
of reliable operation of the device. 

The described principle of signalling and prediction has 
significant advantages in the sense that neither the signalling nor 
prediction requires the introduction of any additional internal 
elements. Usually the performance of these functions relies upon 
special units of the discrete device which require elements having, 
in principle, a reliability as much as one order of magnitude 
greater than the elements which make up the discrete device itself. 

In the design examined above, comprising a structure of 
signal outputs based on actuating devices already having internal 
elements, and assuming that the connections between these 
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Table 3 


*1 

*2 

*3 

*4 

*5 

C„ 

C, 

C 2 

C 3 

Q 

c. 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

1 

0 

0 

0 

1 

0 

1 

0 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

1 

1 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

0 

0 

0 

0 

1 

1 

1 

1 

0 

1 

0 

1 

0 

0 

0 

1 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 

1 

0 

0 

1 

0 

0 

0 

1 

0 

1 

0 

1 

1 

1 

1 

0 

0 

0 

0 

1 

0 

1 

0 

1 

1 

0 

0 

0 

0 

0 

0 

l 

1 

0 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

1 

0 

0 

0 

0 

1 

1 

1 

1 

1 

0 

0 

1 

0 

0 

0 

1 

0 

0 

1 

1 

0 

0 

0 

1 

0 

0 

1 

0 

1 

0 

1 

0 

0 

0 

0 

1 

1 

1 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

1 

0 

0 

0 

0 

1 

0 

1 

0 

1 

1 

0 

1 

0 

0 

0 

1 

1 

0 

0 

1 

1 

0 

0 

1 

0 

0 

1 

1 

1 

1 

1 

1 

0 

0 
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1 
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1 


devices and the sensing signal and predicting devices have 
100 per cent reliability, one would expect that the signalling of 
failure would have absolute reliability in principle. 

In fact, only two mutually exclusive events may occur: 
(a) not one of the internal elements is faulty. Then the actions 
equal to one appear at the corresponding operating outputs 
and at the signal output the action is equal to zero; (b) failure 
of one or several internal elements occurs within the limits of d. 
Then an action equal to one appears both at the signal and 
operating outputs. 

It is noted that achieving reliable operation by means of the 
introduction of structural redundancy according to the principles 
previously presented by the author 3 pertain to the internal 
elements of the device as a whole, that is, both to the actuating 
and the reacting devices. Therefore, with respect to failures of the 
actuating organs, the device retains its ability . to perform 
exactly the control algorithm upon the failure of either one or, 
simultaneously, all of the actuating devices of a given internal 
element for the conditions when these failures are all of a 
single type. 

The described principle of designing signal circuits makes it 
possible to provide separately for signalling the number of fail¬ 
ures greater than d , including those located between the limits of 
d + 1 and d + A. Additional outputs must be added for this 
purpose. This requires that ones be written in the specific rows 
in the appropriate columns of the table of states; namely, for 
signalling failures of elements within limits from d + 1 to d + A 
in the rows corresponding to failures in these limits, and for 
signalling a large number of failures in the rows corresponding 
to unused states. 


It is obvious that the signalling of failures may be not only 
general but also specific, or, for each of the internal elements 
of the device separately. For this purpose one must have for 
each of them an individual output, for which there must be 
written in the columns of the table of states ones for all states 
differing from the basic by the change in value of the correspond¬ 
ing variable. For example, to signal the failure of element X 1 in 
the above case, ones must be written for each first row of the 
sets Ni for the corresponding output. 

Table 3 gives the corresponding values of outputs for each 
of the internal elements. The realization of such outputs pro¬ 
vides, in the event of faulty elements in the device, for advance 
notification as to which of the internal elements is malfunc¬ 
tioning or, with prediction, an approximate indication, per¬ 
mitting timely replacement or adjustment of the element for 
proper action. 

Obviously it is possible to provide not only for signalling of 
failures of individual internal elements but for the separate 
signalling of the nature of these failures as well. For example, 
in Table 4, for the internal element X x examined above, are 
shown the operating states corresponding to failures of the 
typc o—> 1 [Table 4(a)] and failures of the type 10 
[Table 4(b)\ 


X x x 2 x z x 5 

1 0 0 0 0 

110 11 

O) 


Table 4 


X 1 X 2 X 3 X 4 

0 0 11 
0 110 


*5 

0 

1 


In conclusion some of the problems of realizing signalling 
and prediction networks are considered. The circuit of each 
output in the structure of a multi-cycle discrete device must 
contain actuating devices of both internal and sensing relay 
elements. The signal circuits must contain actuating devices of 
only internal elements. Therefore the rational design of the 
structure of a discrete device would be that shown in Figure 2, 
namely, a structure in the form of a certain [1 ,K\ terminal net¬ 
work having at its outputs all the functions of fi and and 
containing the actuating devices of only the internal elements, 
and an [M,N] terminal network containing the actuating devices 
of only the sensing elements. 

As was pointed out above, the functions which realize the 
basic states together with the sets of adjacent states are sym¬ 
metrical with the operating numbers from K — d to K and for 
their realization it is suitable to use so-called ‘threshold’ elements. 
When such elements are used it is advantageous to use the 
structure of the discrete device having a form shown in Fig¬ 
ure 2(b), where the [1 ,K\ terminal network is based on thresh¬ 
old elements according to the number of basic states. The [ M,N ] 
terminal network has the same make-up as that shown in 
Figure 2(a), while the output circuits for signalling and predic¬ 
tion of failures are derived from the outputs of the threshold 
elements by means of their series connection (providing an ‘and’ 
operation) and from circuits corresponding to the function /*. 
The latter may also be designed with the aid of threshold elements 
having symmetrical functions with the operating number K. 

In addition it is noted that, in the case examined above, it is 
most rational from the viewpoint of the simplest physical 
realization of the structure of a discrete device to choose the 
operating levels of the symmetrical functions not from K — d 
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(a) 



Figure 2 

to K but from 0 to d , while simultaneously taking not the 
variables but their inversions. 

In conclusion one should note that the method considered 
previously by the author 3 , as well as everything discussed in 
this report, refer to the case in which the probability of failure 
for all internal elements has a single value, the failures 
are symmetrical (that is, the probability of failures of the 
type 0-> 1 is identical to that of type 1 -> 0), and, in addition, 
failures of individual elements are mutually independent. Con¬ 
ditions differing from these necessitate a somewhat different 
approach to determining the minimum number of elements and 
the distribution of the states. However, the principles of de¬ 
signing signal circuit and of prediction remain the same, with 
the exception that the functions characterizing the basic sets 
and the sets of adjacent states may not prove symmetrical. 

Further Work 


Figure 3). In the papers published up to now it was proposed 
as a solution to use special signalling circuits composed of 
elements whose reliability is as much as one order higher than 
the reliability of the main elements. These circuits have the 
complexity and sometimes even more, in comparison with the 
main circuit. 

The method considered in the paper permits the use, in the 
signalling circuits, of the same elements as were used in the main 
circuits in case of absolute coupling between actuating and 
reacting parts of the relay elements. On the other hand, the 
absence of such a coupling allows considerable reduction in the 



Q 





_ 1 Ti 



— 

X 



Figure 3. A — Inputs; Z — Outputs; C 0 = Outputs for transmitting 
signals; Q = Combinational parts of device; X = Discrete controllers 

complexity of signalling circuits and improves reliability. It 
would be useful, in this case, to apply the method of combina¬ 
tional circuit design proposed by Zakrevsky 9 , whose method 
provides signalling of failures by means of a combination of 
actions at the output of discrete control systems. 
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DISCUSSION 


G. C. Moisil, Academy of the R.P.R., Calea Victoriei 125 , Bucarest , 

Rumania 

We are now working on the problem of finding a generalization of 
Professor Gavrilov’s results for the case of many-valued Lukasievicz 
algebras and for the case of the ring of integers modulo //. 

In fact, polarized relays, latching relays and the real operation 
of ordinary relays, when the intermediate states are considered, reflect 
to three-value Lukasievicz algebras. 

Certain special relays introduce the ^-valued Lukasievicz algebra. 
The rotary switch introduces the ring of integers modulo n. We hope 
to be able to generalize Professor Gavrilov’s results in all these cases. 

Chuan-shan Wang, Institute of Automation , Academy of Sciences , 

Peking , China 

Having read Professor Gavrilov’s paper, which concerns the new and 
important problem of the construction of relay circuits with given 
reliability, I make the following remarks. 

Due to the hazard of relay operation the circuit proposed by 
Professor Gavrilov may have intermediate states which can greatly 
reduce the reliability. For example, the four basic states chosen in the 
example in the paper are 

0 0 0 0 0 
10 110 
0 10 11 
1110 1 

When a transition from the second to the third of these states takes 
place, the intermediate state 1 1 1 l 1 may occur and this state belongs 
to the reception region of another basic state, i.e., 1110 1 (the 
distance between l 1 1 l 1 and 1110 1 being only 1). If one relay 
is damaged beforehand, then the situation will be still worse. This 
phenomenon was first noticed by one of the workers in our laboratory. 
In order to avoid these effects special methods such as the introduction 
of time delays or the use of clock pulses, have to be applied, otherwise 
the arrangement of the basic states, i.e. the construction of the error 
correcting code, has to be considered very carefully. This will introduce 
theoretical and practical difficulties, especially when the number 
of relays is large, and when, at the same time, effectiveness and minimi¬ 
zation are considered. What is the author’s opinion about these 
problems ? 

The second question concerns the practical application of predic¬ 
tion. The author’s ideas of prediction of the failure of relays is very 
interesting. If the arrangement of states is uniform and the reception 
region of each basic state is symmetrical, a decision element may be 
used as decoder and the circuit will be very simple. If, however, the 
circuit is not symmetrical the decoder may be very complex. Since a 
computer must be used in any case, the practical applicability of the 
prediction method is not very clear. 

Finally, I would mention that in one of my papers which was 
published in the Chinese journal Automation in 1958, I reported on a 
study of the symmetry characteristics of error-correcting codes and 


the construction of decoders. The main points of this paper are very 
similar to those in the first part of Professor Gavrilov’s paper. 

M. A. Gavrilov, in reply 

Answering Mr. Wang’s first point on the question of instability of 
inner elements during transitions from one stable state to another 
(hazard phenomena), the suggestion of using relay elements with 
different time delays is not the best one. The problem is to secure 
elementary transfers between sets of states (. Figure A) where fy (j = 
1, 2, ..., n) is a basic state and My (j = 1,2,..., n) are adjacent states. 
There are no critical hazards within these sets. They must be considered 
only between the sets. 



Figure A 


The problem might be solved by the known method of assignment 
of states based on the table of typical assignments suggested by Pro¬ 
fessor Hoffmann (U.S.A.) which provides for independence of in¬ 
stability of element delays. 

The second point deals with the complication of realization of the 
structure of a device in the case where symmetrical functions cannot be 
realized with a single inner element. Principally there are no difficulties 
in the structural synthesis of the combinational part of the relay device, 
which has a certain reliability; see Gavrilov 1 and Zakrevsky 2 . 

In fact the main problem of signalling failure is the reliability 
of the signalling device, which should be an order of magnitude better 
than that of the combinational part of the system. The method sug¬ 
gested in the report is decreasing the complexity of the signal circuits 
and thus making the solution easier. 

Unfortunately I am not familiar with Mr. Wang’s article which 
was published in Automation in 1958, and can only state that I am 
satisfied that the results gained proved his conclusions. 

References 

1 Gavrilov, M. A. Structural redundancy and reliability of relay 

circuits. Automatic and Remote Control. Vol. 2, p. 838. 1961. 
London; Butterworths 

2 Zakrevsky, H. Functional stability of relay circuits. Trud. sib. 

Fiz.-tekh. Inst. Vychis Litelnoj Teknika , Automatica, Teory In - 
formacii (1961) 
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Axiomatization of the Theory of Simplification 
of Combinational Automata 
Gr. c. moisil 


Summary 


The paper shows that the classical method, due to Quine and others, 
of simplification of dipoles with contacts, is a consequence of axioms 
I V and may therefore be applied not only to dipoles with series 
or parallel contacts and to circuits with diodes, but also to circuits 
with triodes, transistors or cryotrons. 

An example is given in section VII. Upon replacing axiom IV by 
IV* the method may be applied to circuits with multivalued elements, 
e. g. relays with real time functioning, polarized relays, relays with 
multipositional armatures and rotary switches. 

Sommaire 

On montre que la methode classique de simplification des dipoles 

avec contact, de Quine et autres, decoule des axiomes I V. En 

consequence, elle peut etre utilisee non seulement dans le cas des 
dipoles avec contacts en serie ou en parallele et dans le cas des 
circuits avec diodes, mais aussi dans le cas des circuits avec triodes 

transistors ou cryotrons. ' .^ 

On donne un exemple dans la section VII. En remplagant l’axiome 
IV par IV*, la methode peut etre appliquee aux circuits avec elements 
a valeurs multiples avec relais fonctionnant en temps reel, relais 
polarises, relais avec armature a positions multiples et commutateurs 
rotatifs. 

Zusammenfassung 


/ (x x ,..., x„) = n [/ (a,,..., a„) u L ai 0,) u ... u L„ n (x,,)](2) 

a 

where 

L 0 (z) = z, L 1 {z) = z (3) 

The formulae are of the following type: 


/Oi* = Or)0 ... 0L an OO] (4) 

a 

where 0 and co represent two operations with any number 
whatever of variables and where expressions such as 


Q Z^ZiCD ... coz r 
1=1 
r 

0 z i = z l O ... Qz r 

i= 1 


(5) 


with r > 1 have a meaning, namely eqns (1) and (2) are eqn (4) if 


I 

II 


co 6 
u 

u 


c *\ .«n 

/(<* ..O 

J ((Xj, ...,0C n ) 


( 6 ) 


Dieser Beitrag zeigt, daB die klassische Methode der Vereinfachung 
von Dipolen (nach Quine und anderen) mit Kontakten auf den 
5 hier angefiihrten Axiomen beruht. Sie kann daher nicht nur auf 
Dipole mit Serien- oder parallelen Kontakten und auf Schaltkreise 
mit Dioden angewendet werden, sondern auch auf Kreise mit Trioden 
Transistoren oder Cryotronen. 

Im Abschnitt VII wird ein Beispiel angegeben. Ersetzt man das 
Axiom IV durch das Axiom IV*, so kann diese Methode auf Kreise 
mit mehrwertigen Elementen angewendet werden. Hierzu zahlen 
z. B. Relais mit realem Zeitverhalien polarisierte Relais, Relais mit 
Mehrstellungsanker und Wahlschalter. 


It is intended to derive a calculus which the axioms have to satisfy 
m order that the simplifying method, given by Quine for the 
II-dipoles with contacts and relays, should be valid. Quine’s 
method has been thoroughly investigated by many researchers 
whose contributions are important, especially J. McCluskey 
and J. Paul Roth. The axioms given here show that this method 
may be used in many other instances, such as that of circuits 
with triodes, transistors, cryotrons, or with three positional 
relays (as in the case of polarized relays or of real operation of 
ordinary relays) and with multipositional contacts (as in the case 
oi selectors and codified relays). 

A. The researches of Quine and of his successors are related 
to the two expansions in Boole series 

/(x 1 ,...,x„)=U/(a 1 ,...,a„)L ai (x 1 ),...,L Kn (x„) (1) 

OC 


It has already been shown that six other expansion formulae 
of the eqn (4) type are valid: 


co 0 


c 




CCn 


The functions 
variables 


III 

u 

T 

/(«!,. 



IV 

n 

1 

/(«„• 



V 

i 

u 

/(a,,. 



VI 

T 

n 

/(«!,- 


(7) 

VII 

1 

i 

/(ai. • 



VIII 

T 

T 

/(Si,. 



T and _L 

are 

Sheffer’s 

functions 

of several 

z, T.. 

• Tz r 

= ^i 

-Zr 



z,!.. 

. -L z r 

= *i 

u ... U Z,. 

(8) 


The interpolation formula of Lagrange in GF (2) is of the 
same type; as a matter of fact there are here two Lagrange 
interpolation formulae for GF (2) (IX and its dual X): 


co Q 


a U On 


IX + ' /( a u ...,cc n ) ( 9 ) 

X + u f(a u ...,d n ) 
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The functions + and + of several variables are defined recur¬ 
rently 


while 


- 1 ••• ••• “h Z n- l) 4 " ~n 

Z 1 -j- ... -\-Z n = (Zi-{- ... + Z n _ i ) -f- Z n 


( 10 ) 


a + /? = a/? u og3 
afj5 = afu^ 


In all these cases, /(a l9 ..., oc n ) is 0 or 1 and eqn (4) reduces 
itself to the function generated by an expression 


X„) = f s Oi, ■ • •, X„) (12) 


where is an expression, yielding the definition: 

I. The expressions are sequences of letters of the following 
form 

Q (z ul 0 ... 9z hmh ) (13) 

h=l 

r r 

If r > 1, £2 is defined by eqn (4). If r = 1, £lt h is / if co is 

/i=T A - 1 

u, *, +, + and is 7 if w is T or 1; in_(13), z i7 will be sub¬ 
stituted by L a (x p ) and therefore by x p or x^. 

II. Between the expressions, a relation of equivalence = may 
take place, satisfying the following conditions if 

then i 

i i 

©<.=©£>,• 

/ i 

Evidently, u,..., + satisfy condition II. 


B. The application of Quine’s method is based on the 
formulae 

x u.x=l 

, . (14) 

lx —x 

Therefore 

z 0 z x ...z r uz 0 z 1 ...z r u h u ... u t 5 

= (Zq U Zq) Z l ... Zy\J tl'U ... ^ t s 

= 1 z,... z r u( ( u ... uf s (15) 

= Z | ... z r u t s u ... u t s 

In order to apply this method, the terms z 0 z 1 ... z r , z 0 z 1 ... z r 
must be brought to be neighbours and the variables must be 
arranged in a definite order. Therefore, it will be assumed that. 

III. Q and O are commutative; that is to say if n is a per¬ 
mutation of the indexes 1, ..., r, then 

Z tc ( 1 * * * ® Z n (r)" Z 1 ® * * * ^ Z r 

(16) 

Z K(i) CO...O)Z n (r) ^ Z 1 0) ... COZ r 

This property allows the expression to take the form 


(z o 0z x 0 ...ez r )a>(z o 0z x 0... Oz r )a>t l a>...CQt 8 


The commutativity is valid for all the operations given as 
examples: u, *, T, 1, +, +. Yet, in order to make the sim¬ 
plification, it is not necessary that all the steps in eqn (15) 
could be made. It is sufficient that 

IV. The following equality be true 

(Z 0 Oz x O ...Oz^toiZoOZiQ ... 0z r ) (Ot x CO ... (Ot s 

= (z t 0 ... 0z r ) cot x co ... cot s (17) 

It is important to emphasize that for all the pairs of opera¬ 
tions co, 0 , from eqns (6), (7) and (9), eqn (17) remains true. 
That is so much more remarkable, since the various steps made 
in eqn (15), such as the associativity of u, the distributivity of *, 
with regard to u, etc. are not valid for some of these pairs 
(I-X) of co, 6 operations. 

C. This first stage of simplification is valid for: 

(a) the dipoles II with contacts, as well in the normally 
disjunctive form (co = u, 6 = •) as in the normally conjunctive 
form (co = % 6 — u); 

(b) the diode circuits, in the same cases; 

(c) the triode circuits, of the two following forms 

co = u, 0 = 1 (form III) 

co = T, 0 = 1 (form VIII) 

(d) the transistor circuits of the eight forms I—VIII; 

(e) the transistor circuits of the form IX, X; 

(/) the cryotron circuits of the following forms 

co = l, 0 = 1 (form VII) 

co = T, 0 = 1 (form VIII) 

D. In the classical case, the following simplification is made 

xyz u xyz u xyz u xyz 

— xyz u xyz u xyz u xyz u xyz u xyz 
= (x u x) yz u (y u y) xz u (z u z) xy 
= yzuxzuxy (18) 

by virtue of the idempotence law 

zuz = z (19) 

To indulge in this type of computation, it is necessary to 
assume that 

V. The following equality is true 

s/ O cos/ Q cos/ x co ... cos/ r 
=s/ Q cos/\co ...cos/ r 

This property is valid for the operations u, *, T, -L, but it 
is not valid for + and +. 

E. It is known that in the classical case, there can be the 
following type of simplification 

xyu yzu xz = xy u yz u xyz u xyz 

= (xy u xyz) u (yz u xyz) 

= xyuyz (21) 
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The problems arising from this type of simplication con¬ 
stitute the originality of Quine’s method. 

A start is made with an expression such as eqn (13) where the 

z ' h haV ® been replaced by L a (x fi ) as in the expressions provided 
by eqn (4). 

An expression of the form 

^ = L ai (x ai ) e ... 0L ar (xj (22) 

is called a simple expression. 

If J are simple expressions 


£*/ OC 


(23) 


provided that 

{L ai (x ai ), • • L ar (x flr )} => {L Pl ( x bl ), ... 9 L Ps ( x bs )} (24) 

where the inclusion is considered in the sense of the set theory. 

It is obvious that, on the basis of principles I-V, it can be 
deduced from eqn (23), that* 


K (*„.) 6... 6L Xr (xj OL , 8l (x bl ) 6... 9L Ps (x b ) 

= -A, ( x a t )^... (x„ r ) (25) 

Since eqn (23) is equivalent to eqn (24), it is easy to deduce 
that the relation oc between the simple expressions is a relation 
ot partial order, i.e. 

ssfccs?/ 


f th ® s,mple expressions Jt (lj)B and Jt (%M with n 
letters, if two of them are contiguous, simplify a letter according 
to the fourth principle and obtain expressions Jt { , m)l , 

. < iihk)t i Wlth n ~ 2 Otters. The operation is reiterated as many 
times as possible and it will yield <f in the following form: 

<$’=Jt i co... coJf t (Dj/ (Uh co... co^/ 0J)t7 co,£ (iJlik)i co... (29) 

' 4m ° ng l a11 these . the name of prime implicant is given 
to if, from 


can be deduced, 


Let 


(30) 

be the totality of the prime implicants. 

° bvi ° US that if ’ in tIle P roce ss of going from eqn (28) 
to (29), the fifth principle is not used, except when strictly 
necessary 

£=B 1 co... coB v ( 33 ) 

but the example of eqn (29) shows that even this expression can 
be simplified to 

S‘=B /li co...coB flg (32) 

which contains only some of the prime implicants. 

For this end perform again the simplification process 
described at the beginning of this paragraph. Starting from 
eqn (28) and considering each let be those 

prime implicants for which 


if sstccS) and S>cc^, then JazJ’ 
if sstocSt and @ozs/, then 

The relation of contiguity will be defined between two 
simple expressions of the form 


At) 0... 6L asi ., (*„ _ x ) 6L Xs _ t (x li + 1 )e 
- 6L °"0L ast + l (x St+1 )6...6L aii (x n ) „ 

S=L /Ji (x I )0... BL^x,^) 6L Ps +i (x Sl + 1 )6 
■■■Bp ii . t 0(x St ~ 1 )9L Pst+i (x St + l )0... 6L Pn (x n ) 

(the missing letters x Sl , ..., x S( are the same in st and S>) by 

K-AI+•••+|a„-y?„l = l (2 

F. It is proposed to simplify methodically the expressic 

$ = CO ... I 


where are simple expressions with n letters x. 

If the simple expressions Jt t and Jt, are compared 
case they are contiguous, if the letters x (third principle) a 
re-ordered, and if use is made of eqn (17) (fourth principle 
a simple expression Jt to) is formed with n - 1 letters. One thi 
obtains several simple expressions Jt to)l , ...,Jt Uj)n with n - 
letters. By virtue of the fifth principle 


1 =jm^co ... 


(U) 


t co ... 


(U)t7 


It can be seen that this equality cannot be written as 
since 6 is not associative (in particular T and 1 are not associativ 


JtiOzBj 

There is at least one among the B, which satisfies this condition. 
Consider the propositions p, = the prime implicant B, appears 
in eqn (32). Since the simple expression t appears in eqn (28) 
the proposition ’ 


^i = Pj l V...VPj 

must be true. 

(33) 

Forming the proposition 


4 

H 

II 

(34) 

By expanding eqn (34) 


P=R 1 v... vR w 

(35) 

where R, are propositions of the form 


... An* ■■■ & Pa c 

which means that 

(36) 

S=B ai co...coB at 

(37) 

However, B ai , ..., B g may be non-different. 

and therefore the 

application of the fifth principle will lead to a simplified form 
of e . 

Such a form corresponds at each R ( . 



G. The simplification problem may be stated not only 
for the canonical forms 

[ L *u(*i ) 6 - A„)] ® •• • © [A rl A,) 0 ... 6 L arn (.!„)] 

but also for other normal forms, such as 

[A„ A/i u ) 9... 0L ctlpl (*, ljtl )] cq 
■■■*lK^)0---eL Xrpr (x Pr j-] 

since, by applying eqn (4), the missing variables can be introduced 
into every simple expression. 
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Classic example 

g — xy uyzu xz = xyz u xyz u yz u xz 
— xyz u xyz u xyz u xj”z u xz 
= xyz u xyz u xyz u xyz u xyz u xyz 

= xyz u xyz u xyz u xyz 

The expression considered is of the form 

g = r /// 1 \j J /( 2 vj ,y # 3 u 
with 

Ji l = xyz 
'/M 2 —xyz 
Ji 3 = xyz 
Ji 4 = xyz 

According to the fifth principle (Ji^ 2 ), (< Ji 2 , 3 ), (^ 3 , 4 ) are 
i 2 ) — xy 

^/ ( 13 , = XZ 
^( 34 ) = .VZ 


New example 

g* = (xl_y)I(ylz)i(xlz) = (xlylz) 
l(xlyiz)l(j/iz)i(xlz) 

= (xlylz)l(xlylz)l(xij;lz) 
I(xljilz)l(xlz) 

= (x 1 y J_ z) _L (x 1 y JL z) i (x _L y 1 z) _L (x _L y 1 z) 
1 (x lylz)i(xij;lz) 

= (x I y J_ z) 1 (x J_ y 1 z) ± (x JL y 1 z) _L (x i. y 1 z) 


«?* = u Ji\ U .//* U ~#4 

* = x 1 y 1 z 
«/^f = x JL y JL z 
— x _L y _L z 
c ^#4 = x 1 y 1 z 

introduced 

y//? 12 ) = x 1 y 
Ji*i3) — X -L z 

<^(34)=y ^ 


The prime implicants are 

B 1 =Ji ( \2) 

B 2 =Ji (\3) 

B 3 = «^(34) 


J/ l ccB l 
Ji 1 ccB 2 
Ji 2 azB l 
•jM 3 oc B 2 
Ji 2 ccB 3 
Ji^ccBi 


u £ 3 
= xy u yz 


_y^( 12) 

^=<34) 

g = B 1 u5 2 u5 3 is eqn (29) 

J/*ccB* 

J/*azB* 

J/$ozB* 

J/tozB't 

J/*ccB* 

JitozBt 

(Pivp 2 ) & 'Pi & (P2vP3) & 'P3 = Pi & P3 is true 

<f* = £*l£ 3 * 

= (xly)l(y lz) 


A parallel is drawn between the classical example exposed 
at the beginning of section V and another similar one. 

To sum up, in order to apply Quine’s method, it is necessary 
that the first to the fifth principles be valid. 

H. On other occasions, the author has drawn attention to 
the fact that multiplicational elements occur in circuits with 
contacts and relays. A few examples follow. 

(a) In real operating conditions, the armature with contacts 
does not change suddenly from the attracted or the repulsed 


position. There exists also an intermediate position, in which 
the normally open contacts as well as the normally closed ones 
are open (the ‘break before make 5 relays) (Figure 1) or else the 
normally open contacts as well as the normally closed ones are 
closed (the ‘make before break’ relay) (Figure 2). 

(b) The polarized relays whose armatures possess three 
possible positions, namely, resting, attraction and repulsion 
positions. 

(c) The codified relays: examples (Figures 3 , 4 , 5) of codified 
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Figure 2 


x 1 /-* 



Figure 5 



Figure 6 


!? 7 Ur ^ 0SItl0nal reIayS ’ taken from the book of Keister-Ritchie- 
Washburn, and the example given by Ivanin (.Figure 6 ). 

(d) The step by step’ searcher or selector. 

To each element a number of contacts can be associated 
namely: 

(/) In real operation of the relays X of the ‘break before make’ 
type there exist normally open contacts (p° (X) and normally 
closed contacts (p 0 (X); in real operation of the ‘make before 
break relays X there are also normally open contacts (pUX) 
and normally closed contacts (X). 

(«) The polarized relays X have contacts for the attraction 
position c/jj (X) and contacts for the repulsion position (Jf)- 
some of the polarized relays also have, contacts for the resting 

position (p 0 (X), but some others, with an unstable neuter lack 
such contacts. 

( : U ] B 15 c °dified relays JT possess several types of contacts. 

w) The selector 5 with v steps has the brush contacts 

(p a (S),..., ( p v ^ 1 (S). 

I. To each ^-positional element, two sets of n elements 
are associated: 

{a) The ring of residue classes modulo n 


with two operations: the addition and multiplication modulo n 
denoted by + and •. 

(b) The n-v alent Lukasievicz algebra 




Ln (39 ) 
with the natural order relation and with the operations 
a u b = max (a, b) 
a n b — min(a, b) 

The Lagrange functions are denoted by L a (.x ) 

L «( a) = l 

4(lS) = 0,a + J 8 ^ 41 ) 

with oc<=f/(„) respectively a 6 L n . The dual functions L (x) 
are introduced with a 

L a (a) = 0 

405 ) = < 42 ) 

There is a Lagrange interpolation formula inf /(») 

/Or, • • O =£/ («i, • ■., a„)L a , (x,) ...L Xn (x„) (43) 
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and two interpolation formulae in L n 

f(x u ...,x n )= u [/(«!, ...,a„)n L Xl (Xi) n ... n L a ,,(*„)] 

a (44) 

f(x l ,...,x n ) = n[/( a u ..,a,)uI„(i 1 )u...uL, j (x > )] 

a (45) 


Giving [LJx)Y- = LJx) (46) 

L a (x) Lp (x) = 0, a =!= (47) 

L x (x)nL p (x)= (48) 
L t (x)+...+L„(x) = l (49) 

L x (x) u ... u L n (x) = 1 (50) 


to co and 0 in eqn (13), the following values can be given 

co 0 


XI + 

XII u n (51) 

XIII n u 

In ^/(w) respectively in L n , principles I, II, III, and V are 
satisfied for the substitutions XT, XII, XIII of co, (9. 

To the fourth principle, must be substituted 

IV*. The following equality is true 

[L 0 (z) Oy , 0 ... 0y r -] co[L l (z)0y l 0 ... 0y r ] co 

... (o\_L n -. 1 (z)0y 1 0... 0y r J co/jco ...co/ s 

= (^(9 ... 0 y r ) tOtjCO ...co/ v 

Principles I, 11, III, IV*, V allow application of Quine’s 
method to multipositional elements. 

In order to have a better understanding of the method one 
should introduce also the notions of formula of structure, 
function of work and functional equivalence in these general 
cases. 
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multi-positional contacts and its application to the study of real 
contacts). Bui sti. Acad. Repub. rom. sec. sti. mat.fiz , 7 (1955) 231. 
See also the author’s work with Ioanin 

9 A supra functionary schemelor cu butoni reali (On the operation 

of circuits with real buttons). Bui. sti. Acad. Repub. Pop. rom. 
sec. sti. mat. fiz. 1 (1955) 33, see also 1 

10 Sur 1’application des logiques a trois valeurs a l’etude des schemas 

a contacts et relais, Act. Congr. int. Automat. , Paris , 14-24 June 
1956, p. 48 

11 Aplicatiile logicei trivalente in studiul functionarii reale a schemelor 

cu contacte si relee (The applications of trivalent logic to the 
study of real operating of circuits with contacts and relays). Bui. 
matem. Soc. sti. mat. fiz. Repub. Pop. rom. 1 (49) (1957) 197 

12 Sinteza schemelor cu contacte si relee in funefionare reala (Synthesis 

of circuits with contacts and relays, under real operating con¬ 
ditions). Bull, matem. Soc. sci. math.phys. Roum. 3 (51) (1959)65 
A volume on the real operating conditions is in the press 
For circuits with polarized relays see 1 and the author’s works 
J3 Sur la synthese des schemas a relais polarises. Bulgarian Academy 
of Science 2 (1957) 121 

14 Sur la theorie algebrique des mecanisme automatiques. Synthese 
des schemas a relais polarises. Ber. int. matem. Koll ., Dresden, 
22-27 November 1955, Aktuelle Probleme der Rechentechnik. 
Deutscher Verlag der Wissenschaft. 1957. Berlin 
For circuits with codified relays sec 1 and the author’s works 
lr> Logica matematica si tehnica moderna. Logicele cu mai multe 
valori §i circuitele cu contacte si relee (Mathematical logic and 
modern technique. The logics with several values and the circuits 
with contacts and relays). Probleme filosofice ale stiintelor naturii. 
1960. ISRS Acad. Rep. popuhi.e Rom ne 
For circuits with selectors, s:e 1 and Gh. loanin’s works 
10 Sinteza schemelor in care intra selectori (Synthesis of circuits with 
selectors). Bui. sti. Acad. Repub. Pop. rom., ser. mat. fiz. 8 (1956) 
489; Automat. Telemech., Moscow 19 (1958) 855 
17 Sur un type de probleme concernant les schemas a selecteurs. Acta 
Logica, Bucharest (1958) 187 


Further Work 

The theory of abstract finite automata has been reduced to the 
two equations: 


X N + l =F(K N ,X N ) 

(52) 

W N =G(K n ,X n ) 

(53) 


where the domains of K , W , X are the sets of inputs, outputs, 
and internal states of the automaton. 

An abstract description of the real automata can be made as 
follows: Suppose the automaton is built with pushbuttons, 
ordinary relays and bulbs. 

The state K of the pushbuttons (which may be operated or 
released) is determining the state H of the contacts (open or 
closed) of these buttons: 

H N = <t> (K n ) (54) 

For each relay vve shall consider three variables: the variable x 
associated to the operation state of the armature, which can be 
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attracted or released; the variable y associated with the current 
in the relay coil (current or no current) and the variables z° and 
z 1 associated with the contacts which may be normally open or 
normally closed. Let X , Y, Z be the variables for the sets of all 
relays. Then the next internal state is determined by the former 
and the current in the former time interval. 

X N+1 =xp(X N ,Y N ) (55) 

The open or closed state of the contacts is due to the position 
of the armature 

Z n = A(X n ) (56) 

Eqns (55) and (56) are characteristic for each secondary element; 


ordinary relay, polarized relay, latching relay, rotary switch, etc. 
The structure of the network yields: 

Y„ = Q{H N ,X N ) (57) 

The elimination of H, 7, Z gives eqn (52). However, the set of 
eqns (54), (55), (56) and (57) permits a better description of the 
automaton. 

The simplification problem can be formulated in this theory; 
it concerns eqn (57) where the function Q is understood as a 
function generated by an expression which is the structure 
formula. 

This problem is solved in the paper. 
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Summary 

In this work a system with a finite number of states is considered. The 
dynamics of the system is unknown. On the basis of the given criterion 
function it is necessary to control the medium in such a way that a 
minimum of the losses expected is obtained. 

It was proved that under these conditions for a certain medium, the 
controlling solutions must minimize the losses only over the current 
step. When the probabilities for the transition of states of the medium 
are unknown the solutions are adopted on the basis of an assembled 
experiment. The results of experiments are intended for the deter¬ 
mination of the actual parameters of the medium. For the purpose of 
this determination the method of ‘reliable intervals’ was used. The 
algorithm for the adoption of controlling solutions is determined and 
the results are presented for the operation of automatic control for 
two types of media with unknown parameters. The results of experi¬ 
ments show that the algorithm used leads rapidly to the optimum 
range of controlling solutions. 


Sommairc 

Dans ce rapport un systeme avee un nombre fini d’etats est pris en 
consideration. La dynamique de ce systeme n’est pas connue.^ En se 
basant sur un critere de performance, il s’agit d’ajuster ce systeme de 
fa?on a minimiser les pertes. 

II est prouvd que dans ccs conditions et pour un certain systeme, il 
suffit de minimaliser les pertes pour l’echelon courant. Quand la 
probability du passage d’un 6tat & un autre n’est pas connue, les 
solutions sont obtenucs experimentalement par superposition. Ces 
resultats sont utilises pour determiner les parametres du systeme. Dans 
ce but, on utilise la methode dcs “intervallcs fiables” (“reliable 
intervals”). L’algorithme conduisant a la solution est indique et les 
resultats sont pr6sent6s pour l’optimalisation de deux types de systemes 
avec parametres inconn us. Les resultats experimentaux mpntrent que 
l’algorithmc utilise conduit rapidement vers l’optimalisation desiree. 


msammenfassung 

)icse Arbeit bctrachtet ein System mit einer endlichen Anzahl von 
'ustiinden, dessen dynamischcs Verhalten unbekannt ist. Auf Grund 
er gegebenen Gutefunktioncn muB man das System so regeln, daB 
lie zu erwartenden Vcrluste zu einem Minimum werden. 

Wie bcwicscn wurdc, erfordern diese Bedingungen ledighch, 
laB die Verlustc nur wahrend des gegenwartigen Schrittes minimal 
verden. Sind die Wahrscheinlichkeiten fur den Obergang von einem 
Sustand des Systems auf den anderen unbekannt, so gewinnt man 
lie Losungen auf Grund von Beobachtungen am System, deren Er- 
rebnisse zur Bestimmung der talsachlichen Parameter des Systems 
iienen. Fur diese Bestimmung wurde die Methode der „zuverlassigen 
ntervalle" (reliable intervals) benutzt. Der Algorithmus, der zur Lo- 
;ung der Rcgelung fuhrt, wird bestimmtund die Ergebmsse fur den 
legelablauf von zwei Arten von Systemen nut unbekannten Para- 
netern vorgelegt. Die Vcrsuchsergebnisse zeigen, daB der benutzte 
Mgorithmus schnell zum optimalen Bereich der Losungen fur die 
Regelung fuhrt. 


Introduction 

In this article a system with incomplete information concerning 
the medium is considered. Problems of this kind are encountered 


in engineering, economics, and in systems of mass maintenance. 
In the systems of control with incomplete information regarding 
the behaviour of the medium, irregular and inaccurate con¬ 
trolling solutions may be adopted, which results in great losses. 
In connection with this the development of such an algorithm 
for controlling solutions, which would rapidly reduce the number 
of inaccurate and costly solutions, represents an important prob¬ 
lem. The system, which will realize this algorithm, may be called 
the automatic system of control. 


Formulation of Problem 

Let U = («!, w 2 , u z ) be the set of actions at our disposal. 
These actions can occur only at discrete moments of time. On 
each step only one action, u(n) e U , can occur. 

Let X* = (xf, x*) be the set of events, which can be 

received by receivers R. The event occurring on the wth step 
will be denoted by x*(n). 

Let X = (x v x 2 ,x s ) be the set of events, which can be 
received by receivers L. The event occurring on the «th step will 
be denoted by x(n). 

In addition there is the criterion function S (x*(«), x(n + 1)) 
which determines for each step the occurring events. This func¬ 
tion is represented in the form of Table 1. 

From Table 1 it follows that events x*(n) = *(« + 1) are 
the events desired. For any other event the ‘penalty’ represented 
by number r must be paid. 

Event x* may be regarded as the request received on the 
given step, and event x, as the realization of that request. When 
the realization is identical with the request there are no losses. 
Otherwise losses r occur. 

Receivers R receive events x* from medium A. The pro¬ 
cesses in medium A which have an effect on received events x* 
may be described only in the form of a probability. In this 


Table 1 


x* in) 

*1 

x 2 


X z 

*1 

0 

r 


r 

x 2 * 

r 

0 


r 






X*t 

r 

r 


0 
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particular problem it was assumed that P (x* =x*) — p* — 1/z 
where i ~ 1, 2, 3, ..., z. This means that at each stage any event 
x '' e X * may occur with an equal probability. 

Receivers L receive events x from medium B. The processes 
of medium B may be influenced by permissible actions u e U. 
Nothing is known about the mechanism of the effect of pro¬ 
cesses of medium B and of adopted action on events x 9 except 
that such an effect does exist. The structure of the mathematical 
model which will be used for the finding of a connection be¬ 
tween the adopted action and the received event x(n) is re¬ 
presented by the matrix for the probabilities of transition 

(Pu) U j,k = 1,2,3,.... 5 z (1) 

where = p (x a -(«)-> x$(n +1)) is the probability of occur¬ 
rence of the event x(n + 1) = Xj when x(n) = *<; u(n ) = a k . 
In the problem considered are slowly changing unknown 
numbers. 

The aim is that, under the above-defined conditions, the 
losses obtained should be at a minimum. This is the general aim 
of action of an organized system. In this system a stochastic 
piocess takes place and, therefore, the mentioned aim should 
be regarded as the realization of a minimum of mean expected 
losses. In connection with this the problem of automatic control 
is to produce on each step such actions for which the mean ex¬ 
pected losses will be at a minimum. 



Figure 1 


The block diagram of an organized system is shown in 
Figure 1. It may be assumed that in the given system there exists 
a series of actions, which solves the basic problem. This series of 
actions, for which a minimum of mean expected losses is ob¬ 
tained, will be called the ‘decisive (determinative) strategy 5 . In 
the given system there may be several decisive strategies. With 
the change in the operating conditions of the system there is a 
change in the decisive strategy. 

The information regarding the behaviour of the medium is 
incomplete in the system. Therefore it is impossible to determine 
directly the decisive strategy. In connection with this a new, ad¬ 
ditional problem for the automatic control is created. It is, thus, 
necessary to control the actions in such a way that the decisive 
strategy is obtained with the minimum of additional losses. This 
additional problem is reduced to the finding of the decisive 
strategy. 

The Solution of the Problem for Known Probabilities pfi 

First of all, automatic control for the known probabilities 
Pm will be considered. For this, the algorithm for the working 
out of the decisive strategy will be determined. 


The mean expected losses will be determined as the losses 
on the IVth step of the path, where N can be as large as desired. 

For the determination of the corresponding algorithm the 
method of dynamic programming will be used. First, the losses 
over one step will be calculated for the following conditions. 

(a) The initial condition for the approaching step is known. 

x*(n) = xf 
x(n) = x i 

Under these conditions the mathematical expectation for losses 
over a single step is determined by the formula: 

r 

v(x h xf,u k )= £ ptjS(x?;x J (n + l)) = (l-p't l )-r (2) 

7 = 1 

(b) The initial condition is given in the form: 

x(n) = x- L 

Pi = “ is the probability of occurrence of any x* e X*. Under 

these conditions the mathematical expectation for the losses 
over a given step is determined by the formula: 

v* (x h u k ) = £ pf (1 - p*) ■ r=~- ■ r (3) 

i = l z 

From (3) it is seen that for the unknown initial condition, the 
mathematical expectation for losses over a step does not depend 
on the probabilities of transition p$. 

On the basis of the method of dynamic programming it is 
possible to write down the following equation: 

V n+N (Xi («), xf (n)) = min < v (x f , xf, u k ) 

Ilk € U ( 

+ X Ph Yj Pu^n + N-l L x j( nJ r l),X*(n + 1)] 1 (4) 

/* = ! 7 = 1 J 

In this equation the events x* (n + 1), x* (n + 2), ..., x* (n+N) 
are given only in the form of probability p* = J. Therefore, the 
second portion of the right-hand side of eqn (4) may be re¬ 
presented in the form: 

ZaT Z K+N-i(Xj(.n+V,x*(n + l)) = ?—^(N-l)r (5) 

h=l j=l Z 

Under these conditions 

K+n( x i 00 . x * (n)) = min j (1 - p 1 -,) ■ r +Z_!(IV -1) r i (6) 

uueuL z J 

From this it is evident that the optimum solution is that 
solution which gives the minimum value for the mathematical 
expectation of losses over a step and this is clear intuitively. 
For the same probability of occurrence of event xf e U over 
each step, there is no point in planning the actions over 
N steps. 

On the basis of the given reasoning, the following algorithm 
for the operation of the decisive strategy is adopted. For each 
step such actions are adopted, for which the mathematical ex¬ 
pectation for step losses is at a minimum. 
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Solution of Problem for Unknown Values of p§ 

For unknown probabilities it is impossible to apply the 
algorithm determined above. It is necessary to develop a new 
algorithm in order to find a decisive strategy. One of the methods 
for finding this is to use the information regarding the medium 
obtained during the time of operation of the system, and the 
gradual approach to the unknown strategy. 

The results, obtained during the time of operation of auto¬ 
matic control, will be represented in the form: 

Pf) ij,k = 1,2,3,. ,2 (7) 

\nijj 

where m f is the number of adopted actions u k for the initial 
condition x, h and is the number of obtained transitions from 
Xi(n) to Xj (/? + 1), for m\ experiments. 

These results will be used for the determination of unknown 
probabilities p§. The determination of the unknown values 
of pj$ will be made by means of confidence intervals. For each 
value of v ] ym ]l l it is possible to calculate the confidence interval 
(where P$ n is the lower limit, and P$ B the upper 
limit, of the interval. 

The limits of intervals may be calculated from known ex¬ 
pressions or they can be obtained from tables 2 . 

The confidence interval determines the set of the hypo¬ 
thetically possible actual values of p-j. With a high degree of 
reliability it can be assumed that the actual value of probability 
p$ will be found in the above-defined interval. 

Let the initial condition for the approaching step be: 

x*(n) = xf ) 


x(n) — Xi 


vfl 


m ; 




UB. 


!) 


( 8 ) 


The working out of the decisive strategy represents the general 
problem of the system which consists in the control of actions. 
From this it follows that for a given initial condition it is neces¬ 
sary to choose action u h e U for which p$ is at a maximum. 
However, since one knows only the confidence intervals, it is not 
possible to make a direct choice. In connection with this the 
following algorithm for the choice of action u ]c e U, is adopted: 
to choose such u lc e U actions for which, for a given initial con¬ 
dition, there is a hope that probability pft has the maximum 


value. This is identical with the method based on the choice of 
an action, for which there is a hope that the expected losses 
over a single step will be at a minimum. 

It should be pointed out that the upper limit of the confidence 

interval P£ B where Ic = 1,2, 3,. ,2 represents the basis for 

the choice of the action. From this it follows that it is necessary to 
choose such values of u u , for which the upper limit of the con¬ 
fidence interval has the maximum value. The result of the action 
will either confirm the correctness of choice or, in the case of a 
negative result, decrease the upper limit of the interval, which in 
the following intervals gives the possibility for the choice of 
another value for u k . This method guarantees a sufficiently 
quick convergence of the actions being chosen towards the 
decisive strategy. 

Example 1 —In the given example the set of events X* con¬ 
sists of three events (*?, x*, xf). Medium B is described by 
means of graphs, shown in Figure 2. The results of actions of 
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the system are shown on the graph (Figure 3). On this graph the 
eviations of the actual actions from the decisive strategy are 
seen. This system was investigated. 

The results obtained indicate a rapid convergence of the 
actions towards the decisive strategy. 



Example 2—-In this example the behaviour of medium B has 
changed. Medium B is represented by the graph of Figure 4. 
The results obtained are shown in Figure 5 . In this case also, a 
rapid convergence towards the desired strategy is obtained. 




1 Feldbaum, A A. Information storage in closed systems of auto¬ 
matic control. Izv. Akad. Nauk SSSR, OTN, Energ. i Avt. 4 (1961) 

- Yanko, Ya. Mathematical-statistical Tables , 1961 

Bellmann, R. Dynamic Programming. 1957. New York 

4 B 1955 R and M ° STELLER ’ F ' Stoch astic Models for Learning. 


DISCUSSION 


J. Sklansky, R.C.A. Laboratories, Princeton, New Jersey, U.S.A. 

The optimal decision strategies discussed in the paper are optimal with 
respect to performance, but not necessarily with respect to construction 
and maintenance costs. In fact, many so-called ‘optimal’ decision 
strategies can only be realized by digital computers. This situation is 
especially severe in the case where the transition probabilities are un¬ 
known, since the reliable intervals’ p, i.e., the range of likely values 

P ijt must be recalculated at every step. 

To overcome this shortcoming, the following procedure is suggest¬ 
ed. Choose simple, easily constructed, easily maintained decision 
strategies then determine the class of plant-environment combinations 
for which each of these decision strategies can be expected to yield 
goo per ormance, taking into account the designer’s a priori igno- 
ranee of the parameters of the plant and its environment. 

This procedure has been applied to a simple problem in adaptive 
signal detection’, illustrated in Figure A. A binary information source 


Binary 


Plant 



Figure A 

A J~f . COI " bhlat i on of the Paper's ‘medium A’ and ‘receivers PC) emits 
Where the , Set of events x * consists of just two events: 0 and 1 
Wherever possible, the mathematical notation here is the same as in 
he paper. The plant (medium B’) consists of a noisy channel C a 
subtractor S and a sign detector Z>. The decision strategy is incor¬ 
porated iri the controlling medium’ E. The decision strategy is very 
simple, when the plant’s guess 

u(n + i)J u ^ +1 if 

C 3 if u ( n ) = 3 

When x (// + 1) = 0 and x* (/?) = 1, then 

(.1 if u (n) = 1 


This is called ‘simple incremental’ decision strategy. The resulting 
Markov chain model of the closed-loop process is specified by the 
following stochastic matrix: 


1 

2 3 

1 + a ) 

~(l+ix) 0 

2 

y(l + a) 

0 

2 ( l+a ) 1- ~(l+oi) 


In this matrix the states are the values of the control actions u. The 
variable p is a parameter of the information source, and a is a para- 
meter of the plant. F 

Jl^ Se f s Performance index, denoted by z („), is defined as 
probability of a correct guess. Symbolically, 


z(n) = Pr[ x (n) = 


= *‘‘ 0 - 1 )] 


Suppose ‘good’ performance is defined as z(<x) > 0-8. Using these 
efinitions one may compare the effectiveness of the simple incremental 
strategy (the closed-loop’ operation) with an ‘open-loop’ operation 
m which u (n) = 2 for all n. To do this the regions in the ga plane 
corresponding to z(oo)> 0 -8 for the open-loop and closed-loop 
operations were computed. The results are shown in Figure B. The 
good regions in the got plane, corresponding to z (oo ) > 0-8 are shown 
shaded. Note that the closed-loop operation yields a substantially 
larger good region than that of the open-loop operation. 

When the transition’ probabilities of the plant are unknown it is 
possible, as indicated in the paper, to estimate ‘reliable intervals’ for 
j n .. IS . n r ranCe rectangIe ’ in the Parameter space, as indicated by the 
dotted lines in Figure B. It is suggested, that the designer should have 
at his disposal an ‘atlas’ of performance contours similar to those in 
Figure B. The designer will then be able to choose the least expensive 
decision strategy whose ‘good’ region covers—or almost covers—the 
ignorance rectangle of his particular problem. 

This approach to the design of decision strategies promises to 
reduce substantially the construction and maintenance costs for a 
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(X 

Open-loop 



Figure B 


given performance requirement, taking into account the a priori un¬ 
certainties of the environment and the unreliability of the plant. 

Reference 

1 Sklansky, J. A Markov chain model of adaptive signal detection , 
presented at the 1963 Bionics Symposium, March 19-21, 1963, 
Dayton, Ohio, U.S.A. 

S. Paszkowski, in reply 

Mr. Sklansky has proposed a very interesting approach to the solution 
which I investigated in the present paper. I agree that the development 
of rational solving strategies and the analysis of the performance of a 
system controlled by this solving strategy is of great importance. I 
have proposed one of the possible algorithms for developing a solving 
strategy. 

Y. W. Eveleigh, General Electric Co., Syracuse , N.Y., U.S.A. 

The author presents several interesting examples involving straight¬ 
forward application of probability and dynamic programming prin¬ 
ciples. Although the applications are perhaps unique, the principles 
upon which they are based are well known. 

I wonder if the author has considered, in addition to his single 
dimensional example with first-order memory (Example 2), any prob¬ 
lems with higher-order memory in which the transition probabilities 
depend upon present system state and also on the previous state and 
perhaps others prior to that? This is equivalent to dealing with more 
elements in the state vector and considering only first-order memory. 
This should be an interesting problem. Also, when the transition 
probabilities are unknown and must be measured as the solution 
proceeds, there is a potential solution instability problem. The prob¬ 
ability values may be changing more rapidly than the system is able 
to follow using only the information obtained from relatively infre¬ 
quent sampling of response. This point should be mentioned in the 
paper. If the rate of change is very slow, convergence is assured, as¬ 
suming, of course, that the measurements of state are reliable. Finally, 
the significance of Figures 3 and 5 is not completely clear to me 
from the text. I would appreciate a further comment on their 
meaning by the author. 

S. Paszkowski, in reply 

I should like to point out that in the studied problem medium B has 
memory, i.e., the probability for the occurrence of a definite event 
x(n 1) depends on the state a (n). 


An intersting remark by Mr. Eveleigh is the possibility of in¬ 
stability. I have worked on this problem and the results obtained are 
concerned with the dependence of the quantity of memory on the rate 
of variation of transform probabilities. 

Concerning the second part of the question, it must be said that 
in Figures 3 and 5 differences between the number of solution, which 
at the same time must be also the number of the realized solution, 
are plotted. 

Chuan-Shan Wang, Institute of Automation , Academy of Sciences , 

Peking , China 

I should like to ask Professor Paszkowski the following questions: 

(1) What is the practical significance of his paper? 

(2) Has the author constructed a practical model of such automata, 
and what are his experimental results? 

(3) How did the author compute the results given in Figures 3 and 
5 of his paper? 

(4) In my opinion, problems with unknown values of p% can be 
solved by known principles of modern statistics theory of communi¬ 
cation. When the a priori probability is uniformly distributed it will 
yield the same results as obtained in this paper. What is the author’s 
opinion? 

S. Paszkowski, in reply 

(1) The algorithm in question can be applied to control such complex 
systems as transport systems and others in the case when the prob¬ 
ability of the transition from one state to another is unknown. 

(2) An automaton operating according to a given algorithm has a 
form of programme for a digital computer. This computer was work¬ 
ing with an experimental object. The results obtained confirm the 
rapid convergence towards the solving strategy. 

(3) The results presented in Figures 3 and 5 were obtained by 
means of a computer. 

(4) Unknown probabilities p% can be estimated directly from the 
uniform distribution of a priori probabilities, but it is necessary to 
improve these estimates by means of experiments. The method of 
uniform distribution of probabilities a priori gives no possibility of 
controlling such experiments in order to obtain the improved estimates 
of necessary probabilities. 
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THEORY OF OPTIMAL SYSTEMS 


The Synthesis of Optimal Regulators 

A survey by A. M. LETOV 


Introduction 

In a short report one cannot describe the content of the huge 
flow of literature that has been devoted to optimal control. The 
compilation of a survey has been made still more difficult by 
the fact that eminent mathematicians of many countries are 
taking part in the investigation of optimal control processes. 

Outstanding successes have been achieved in their efforts to 
make the reasoning schematic and as general as possible, and 
to subordinate it to comprehensive laws of mathematical forma¬ 
lism. The methods of the maximum principle and of dynamic 
programming serve as a brilliant illustration of this. Thus any 
attempt to revise the theory of optimal control will inevitably 
verge on an assessment of the established mathematical canons, 
which the author dares not undertake. 

These difficulties become insuperable when one tries to shed 
light on the historical aspects of the theory, and these will not 
be considered. 

Hence, the author decided simply to set out his own views 
on the contemporary state of the problems in synthesis of optimal 
regulators for continuous systems. 

In forming these views, the author has not relied only on 
reading the literature, but has sought support in the most in¬ 
structive discussions he has had with Academician L. S. Pon- 
tryagin, Professor A. I. Lurye (corresponding member of the 
U.S.S.R. Academy of Sciences), Professor Ye. A. Barbashin, 
Professor V. I. Zubov, and Professor N. N. Krasovskiy. 

Setting of the Fundamental Problem 

Programming of Brachistochronic Motions 

There are two classes of optimal control problems. The first 
includes problems in programming optimal motions which have 
extremal properties that are specified in advance. The program¬ 
ming problem for deterministic systems is formulated as follows. 

The differential equations for the controlled plant are given: 

Here x = X(x ,p,t) (1) 

x {x x ,..x n } is the state vector, ,u is a scalar control function*, 

* For the sake of simplicity one considers only plants with a single 
actuator. The following discussions can be generalized without any 
difficulty in principle to plants with a number of actuators. 


X {Yj,..., X n } is the generalized force vector, defined over 
the region N (x, fi) > 0 characterized by the inequality 


R(x 9 p, t)> 0 


The system (1), (2) is defined for te [ti , //]. 

The boundary conditions are given, which are expressed 
symbolically as 

(/./) = 0 • (3) 


A certain class of comparison functions is chosen (e.g. x the 
class of smoothed-segment functions and p the class of con¬ 
tinuous-segment functions) and one determines for this the opti¬ 
mizing functional 

= t) (4) 


Functions x and p satisfying (1), (2) and (3) are called permis- 
sible. 

The programming problem is to find from the permissible x 
and fjb those functions which make the functional (4) a minimum. 

It is assumed that X, R, and also conditions (3) are such 
that the required solution exists uniquely and has the form: 


x=x*[f, (i,/)], n = (i*\t, (/,/)] (5) 


The motions (5) can be called brachistochronic, since they are 
expressed in parametric form, i.e. just as the solution of the 
famous problem propounded by Bernoulli was 267 years ago. 
It is material to emphasize that the programming problem is a 
transferred problem of the particle dynamics of the controlled 
plant, where the forces and motions are partly given and it is 
required to find those that are missing. 

As such, this problem lies outside the field of the direct scien¬ 
tific interests of specialists in automatic control. In fact, to set up 
a mathematical model of the problem (to know the forces) re¬ 
quires a profound knowledge of the physical laws to which the 
plant (1) is subjected, and a precise definition of the ultimate aims 
in controlling it. The latter is the prerogative of scientists and 
engineers in the field of applied science—particle dynamics—to 
which the plant belongs. It is they who are capable of evaluating 
the extent to which the solution (5) actually meets the objective 
assigned. 
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The automation expert is called in to indicate the method 
and means for actually achieving the programmed motion , and 
handles it as a premise. Of course he does not lack interest in the 
model itself, in its properties and the peculiarities of solution (5). 

The focus of his scientific interest lies, however, in the 
different problem of optimal control. 


Synthesis of Optimal Regulators. Basic Problem 

The problem begins as follows. Let the solution (5) be called 
an unperturbed motion in Liapunov’s sense. It is easily verified 
that solution (5) cannot be achieved, if only because state T (3) 
cannot be precisely realized. Hence the actual motion will be 
described by the functions 

x = jx* + t?, /i = /i* 4-C (6) 

where y {>/ l5 ..., y n }, f is a Liapunov perturbed motion, while 
y (0), £ (0) is the initial perturbation*. The perturbed motion 
satisfies the equations 

q = Brj-\-m£ + E(rj, £, t) ( 7 ) 

The right-hand side of (7) are some form of series expansions 
of Xjc. In particular they may be Taylor expansions, in which B , 
the square matrix \\bk a \\ = ||0X&/0x a ||*, and m, the column 
matrix ||m*|| = ||0X&/0/*||*, are determined for the motion (5). 

Note that in setting up the basic problem it is not necessary 
to assume that the motion (5) is a solution of some variational 
problem. The motions y, f are constrained by the inequalities 

jR (x* + rj, p* + Z,t)>0 (8) 

For any y (0), perhaps constrained by some condition 

£>*(0)<U (9) 

where A is a given number, and for a given function £ (0), the 
relations (7) and (8) define what is called in automation a 
transient process. 

One of the main problems of an automation specialist con¬ 
sists in the control of the transient process. He has to find the 
technique and means for damping this process in what is in some 
sense the best manner. In particular it is desirable that for t — T 

Zni(T)<5 00) 

where <5 is a number chosen in a given way. 

In order to achieve this, one may have at ones disposal a 
choice for the function £. Hence a certain measure is introduced 
for assessing the deviation of the actual motion (6) from the 
programmed one (5). 

Let this be the functional 


J(0 = r 


W (rj, I, t) d t 4- sw [jj (T), £ (T), T] (11) 


Here W and w are certain non-negative and sufficiently smooth 
functions of their arguments, while r and s are non-negative 
numbers. 

The functional (11) is the evaluation of the response of the 
transient process and of the final state of the system. In order to 
complete the setting of the basic problem , one needs to choose the 


* Take also t% — 0, // = T. 


class of comparison functions for which the functional (11) is 
determined. Mathematically it is convenient to choose this class 
as broadly as possible. The range of choice may, however, be 
limited by considerations of a practical engineering nature. 

The simplest solution is formulated without any allowance 
for the form and characteristics of the modern technical equip¬ 
ment on which the achievement of the required solution will be 
based. Hence after formally defining such a class G of these 
functions (e.g. £, rj to be smoothed-segment functions) the follow¬ 
ing basic problem on the search for an optimal regulator is 
stated. 

Fundamental Problem —For the class G of permissible com¬ 
parison functions it is required to determine a differential equa¬ 
tion 

F[...<U,M] = 0 (12) 

expressing the control law, such that a regulator constructed 
according to law (12) and connected to the plant (7) should 
ensure a minimum of functional (11) for all motions starting 
within (9). 


Note 1 

The inequality (10) is not, generally speaking, a boundary 
condition. The expression S y\ (T) is itself a functional, and (10) 
merely expresses one’s natural wish to bring the current point 
of the system into a certain limited domain of its state f. In the 
particular case where £>* a , m k = const, or are periodic functions 
in /, the ideal substitution T = co is permissible for functional 

(ID. 

Then one may put <5 = 0, and condition (10) will mean 

rj fc (oo) = 0 (fe = l,...,n) (13) 

i.e. the regulator (12) must in addition ensure the asymptotic 
stability of the system. The latter is a concomitant of the finite 
nature of functional (11) for T = co. 

If nevertheless T < co, then by putting 


T-t. 


-1 


(14) 


the basic problem of synthesis is brought into the form: 
T 


y = 


J=r 


(1+*| 

r T 


j(Bri + m£ + E(ri,Z, t)) 

T 

C, £,t)dT + sw 


( 15 ) 


o(i+*r 


Hence for finite T also one may speak formally of the stability 
of the system as ensured by the regulator 


*” dr 




( 16 ) 


This will essentially signify the monotonic damping of a certain 
positive definite function V (rj, £, t) for t e [0, T]. 

In this case one can ensure that there exists a number 6 < A 
for which inequality (10) is satisfied. Of course such a transfor¬ 
mation is otherwise of a formal nature, and does not essentially 
supersede the investigation of solutions of (15) and (16). 
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Note 2 

If IF contains no derivatives of £, then (12) degenerates and 
becomes algebraic: 

F l€ >*!>*] (17) 

One can say that this defines an ideal regulator , i.e. one 
which requires an energy source of infinite power. 

Note tha t formally, by suitable substitution of the variables, 
the basic synthesis problem may sometimes be reduced to the 
achievement of a regulator of type (17) for all W. Given below 
is such an example. 


gramme mechanism 1. Such a mechanism can always be con¬ 
structed using a clock. 

The signals from 1 pass to the summing device 2. They give 
rise to a programmed deviation of the controlling device p*; its 
additional deviation £ is generated by the regulator 3, which 
uses information rj (t) on the actual state of the controlled plant. 

This information is detected by a suitable collection of meas¬ 
uring instruments operating according to the algorithm contain¬ 
ed in (12). This layout expresses the general concept of modern 
automatic control theory. 


Note 3 

In the statement of the fundamental problem it is assumed 
that the mathematical model (7) possesses the property of ‘com¬ 
plete controllability and complete observability’, or the less 
powerful property of‘complete stabilizability’ (see under ‘General 
Control Layout’). 

An example of the synthesis problem showing how the re¬ 
quirement of complete observability’ may be reduced is now 
given. As for the requirement for ‘complete controllability’ or 
‘stabilizability’, it appears to be so natural and self-evident that 
there cannot be any question of abating it. 

. In fact > the model (7) could surely only prove uncontrollable 
if some essential terms had been left out of account in eqns (1) 
or if the programme (5) had been chosen unreasonably. 

Of course nobody can ensure against such an error, but this 
once more underlines the wisdom of referring the programming 
problem to the competence of a dynamics specialist. The ‘con¬ 
trollability criterion’ or the ‘stabilizability criterion’ presents him 
with an effective and rigorous means of finding out whether the 
model he has constructed is adequate for the given controlled 
plant. Only when the criterion is satisfied can he pass this model 
over to the automation specialist for further investigation. 

General Control Layout 

Let it be assumed that the control law (12) is known. The 
general structural layout of the control system is shown in 
Figure 1. 

Here the brachistochronic motion is contained in the pro- 



Figure 1. Overall control layout 


Possible Allowance for Various Characteristics of Modern Tech- 
nical Equipment 

Another statement of the problem of synthesis of optimal 
regulators relies on the following considerations. 

For example, whatever the form of the optimizing functional 
(11) and the nature of eqn(12), one can only achieve this equation 
by the use of a servomotor, whose speed is limited and depends 
appreciably on the external load. 

In such cases it may be better, for example, to take account 
at once of the equation 

C =/(<?) *A(w) (18) 

for the speed of the servomotor, if it is hydraulic. Here \l/ (w) is 
the prescribed function 

*A = Vw (19) 

for reversible servomotors and 

1 w> 1 

^W=V w ’ 0<W<1 (20) 

0 w<0 

for non-reversible servomotors. 

In the general loading case 

w = 1 - (P 2 £ + + r £) sign a (21) 

Here F, Q and r are given constants. 

As for the function f(o), it is only defined to the extent that 
it belongs to a certain class B of functions characterized by the 
properties: 

(1) /O)=0, <7/O)>0 for 0|<O 

(2) /O)=0 for j a\>a (22) 

(3) I/O)! </ 

Here o and /are given positive numbers. 

The class r/, f of snioothed-segment functions that satisfy 
eqns (7) and (18), conditions (9) and limits (8) and (22), can 
be described as permissible. 

Problem: For the class of permissible functions, complete the 
construction of eqn (18), i.e. determine accurately the function 
f (cr) sB and the switching function 

<r=<r(r ]l ,...,n n ,£,t) (23) 

that make the functional (11) a minimum for all motions starting 
at (9) and make the system (7), (18) stable. 

The problem may be formulated similarly to take into ac¬ 
count the characteristics of an electrical servomotor or of the 
measuring equipment used in the regulator. 
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Selection of the Optimizing Functional 

The selection of the optimizing functional is made on the 
basis of the view as to what is ‘good’ and what is ‘bad’. Beyond 
doubt this question should be resolved in the light of experience. 
In a number of particular cases one considers the functionals 

" T 

W(n 9 tf 9 t)dt (24) 

J o 

which assess the extent of smoothing of the perturbed motions 
??, i along brachistochrones. The presence in W of suitably 
chosen expressions in | and has the aim of minimizing the 
magnitude of (24) with minimum energy consumption by the 
regulator. 

In other problems—terminal control problems—one studies 

the functionals r rr\ s/ta mi oc\ 

w[^(r),c(T),T] 25) 

which represent the magnitude of deviation of the system from 
its terminal condition (/) at the instant t — T. 

So far only quadratic functionals have been studied. In all 
instances no rigorous grounds are given for the choice of the 
functions W and w. This choice is made to a considerable extent 
on an intuitive basis, and constitutes the first experiment in the 
construction of a theory of the synthesis of optimal regulators. 
From the formal point of view, the functional, if it is previously 
chosen, may be considered as a postulate which determines the 
nature of the control theory deriving from it. Hence, just as 
happens in geometry, each functional assumed a priori will have, 
corresponding to it, its own theory of optimal regulator design. 
Just as from all the possible geometries at the outset of its 
development mankind chose for its first science of measurement 
the simplest and intuitively closest geometry of Euclid, so today 
the theory of the synthesis of optimal regulators is being con¬ 
structed round the quadratic functional as the simplest and 
intuitively closest. 

There are also other more prosaic yet important reasons for 
investigating quadratic functionals. 

As will be seen, for example, the well-worked-out technique 
for solving problems of regulator synthesis as optimized by 
quadratic functionals leads one directly to the solution of the 
same problems in which the functionals (11) are appreciably 
more complex. 

Regulators which are Optimal for Response Speed 

If one puts <5 = 0, rW = 1, and s — 0 in (10) and (11), 
one arrives at the problem of the rapid-response regulator. 

Though this problem fits into the general framework of the 
basic problem, it still occupies a somewhat special place which 
justifies its separate treatment in what follows. 

Progress Achieved in Solving the Basic Problem 
General Situation 

The synthesis problem is not as deeply rooted in science as 
the programming problem. It originates in the demands of 
modern engineering, and is caused by the historical course of its 
development. 

Engineers, by relying on experience and intuition, have de¬ 
signed (as far as they were able) regulators first of type (17) and 
then of type (12), mainly in order to maintain required steady- 
state operating conditions in controlled plants. 


Their intuition has led to the discovery of two basic principles 
of control: those of control by displacement (r) and control by 
perturbation (/). The discovery of the feedback principle and its 
wide application in practice has played an immense part in the 
development of automatic control systems. 

Essentially, science is now occupied in seeking merely more 
accurate and more complete forms of these principles. The 
simplest example of such an improvement is the theory of 
observability which is now being developed; in more complex 
cases, in accordance with Wiener’s cybernetic doctrine, one is 
endeavouring to find comprehensive forms of these principles in 
systems with adaptation. 

In itself the wish to discover optimal forms is not so much 
praiseworthy as necessary, in view of the extremely limited 
material resources at the disposal of mankind. 

One is bound to think of the future generation and to expend 
ones resources as economically as possible. 

There are a number of instances, few but nevertheless im¬ 
pressive, in which an effective solution of the synthesis problem 
has been or can be obtained. These will now be described. 

First, however, the characteristics of the methods employed 
will be briefly dwelt upon. Here reliance is placed on the 
evidence existing in the literature for showing what these 
methods have been able to yield. 

Sincerest apologies must be offered to mathematicians for so 
primitive an approach to the assessment of the results of their 
creative activity. For those occupied in the application of mathe¬ 
matical methods such an approach is inevitable and is justified 
by the desire to obtain as soon as possible in analytic form an 
expression of the concrete content of the synthesis problem. 

The assessment of the potentialities of these methods re¬ 
mains, however, the province of the mathematicians themselves. 
The most that one can indicate here consists merely of an as¬ 
sembly of perhaps inadequately justified guesses. 

Methods for Solution of the Synthesis Problem 

The first and simplest synthesis problems were solved by the 
techniques of the calculus of variations. These techniques have 
been extensively developed in the works of the mathematical 
school founded by Pontryagin, and they have been given ele¬ 
gant expression in the form of the general theorem known as 
the maximum principle. 

The maximum principle has made for improved understand¬ 
ing and solution of complex modern applied variational prob¬ 
lems defined in closed spaces. Variational techniques also possess 
other important advantages that do not exist in other techniques. 
For example, they enable one to establish the following fact 
quite simply. If the function W in (11) contains a derivative of 
f of order m, then the eqn (12) will also be of order m. The 
practical application of these techniques however runs up against 
difficulties, which consist in the need to integrate the equations 
of the variational problem. The difficulties of integration are 
increased by the fact that n auxiliary Lagrange multipliers are 
normally employed in these techniques. The latter are com¬ 
pletely unconnected with the essential content of the problem, 
and do not enter at all into its statement. 

On this path the famous two-point boundary problem stands 
as a fortress that has been attacked time after time but never 
conquered. In those rare instances where it becomes possible to 
express the multipliers 2 in terms of the variables 97 , f and /, the 
variational techniques work successfully. 
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The author, however, is not so bold as to claim that success 
is close at hand, in many other cases where the application 
of variational techniques appears both rigorous and legitimate. 
In a number of particular cases the use of the method of 
dynamic programming seems more attractive. 

If one can make use of the definition of ^-functions, then 
the method of dynamic programming leads one to the investi¬ 
gation of partial differential equations in the function 


F OhO, £(0,0 = 


t 


[rW + s<r(7 — T) co] d t 


(26) 


This equation serves as a mathematical expression of the 
Bellman optimization principle. 

The optimization principle is extremely broad, and the method 
of dynamic programming which is based on it is applicable 
to the investigation of optimal processes of very broad scope. 
Of course this method introduces quite a number of difficulties 
of various sorts. For example, the application of the method to 
the solution of the basic problem is limited in extent to functions 
V which are continuous and smoothed-segment in rj and |, and it 
is impossible to find out in advance whether one is dealing with 
these cases. Fortunately in its application to this particular 
branch of science the dynamic programming method is closely 
related to Liapunov’s direct method of determining V functions 
to solve the stability problem. The link is as follows. As a rule 
it turns out that the optimization principle is satisfied by a cer¬ 
tain set of optimizing functions V ( rj 9 |, t). However, only 
those functions of this set that are Liapunov functions for 
the closed system will provide an actual solution of the prob¬ 
lem. 

Regulators working according to such functions are optimal 
and also give the system closed-loop stability. 

It follows that the Liapunov method enables one to select 
the required solutions out of those that are provided by the 
Bellman optimization principle. This combination of the 
two techniques also makes possible an effective solution of the 
synthesis problem. It assumes a special importance also because 
in the actual statement of the synthesis problem there is a great 
need for Liapunov’s rigorously-based and fundamental concept 
of perturbed and unperturbed motions. 

Essentially this concept appeared spontaneously as a basis 
for the development of classical control theory, while now it is 
also becoming the foundation on which modern control theory 
is being built. 

It is thought that the Bellman-Liapunov method has its 
widest possibilities in those cases where some form of successive 
approximation technique is applied to it. One can now discuss 
the results obtained by this method. 

The fundamental equations of the method set out in the form 
of sufficient conditions for optimization are now given. 

Let the equations of perturbed motion be: 


(27) 

These are defined for \\rj\\ < R, re [0, go]; R < 0 is a given 
number. 

By choice of the differential eqn (12) in f it is required to 
minimize the functional 


W (rj, c, z) dz 


(28) 


such that the motion rj = 0 shall be asymptotically stable relative 
to initial perturbations from the region ||^ 0 || < R 0 < R. It may 
happen that R 0 = R = nc . If there exists a function V (rj, t) and 
a function |° satisfying eqn (17), such that: 

(1) they are defined in the region \ \rj\ \ < R; 

(2) the function V (??, r) is positive definite; 

(3) at the point rj = 0 the function V assumes an infinitely 
small upper bound. 


(4) Sup [ V ||tj |i < Rq] < inf [ V, || q || = R] 

(5) The derivative V calculated along the trajectories of the 
system as closed by the regulator (17) satisfies 

(FV=-(F|>,£ 0 ,t] (29) 

and the optimization equation 


(F)jo + W(r,, £°, T) = Min [(F)* + W (r,, C, z)] (30) 

Then £° is an optimal control, and 

€ 

F07 0 ,0) = Min f“ W (rj, £, z) dz (31) 


is valid. If eqn (17) has been found, then the stability of the sys¬ 
tem is ensured by conditions (1)—(5), since the V thus deter¬ 
mined is a Liapunov function, while the optimal nature of the 
regulator follows from eqn (30), which expresses the Bellman 
optimization principle. 

For T< oo, the principle as formulated retains its meaning 
provided that in defining the function V one brings in also the 

boundary condition T _. _ 

V(ri,T) = sw (32) 


and treats the word ‘stability’ in the sense explained under the 
heading ‘Synthesis of Optimal Regulators’. 

It is not necessary to dwell on the characteristics of the 
functional analysis techniques and the various types of approx¬ 
imate methods that have been successfully applied to the solution 
of a number of variational problems. 


Synthesis of Ideal Regulators 

Stationary Linear Systems . Problem 1 

Statement of the Problem —Consider the stationary linear 
systems (7) {S k s= 0), defined for p* = 0, T = co and 

ld<? (33) 

where f is a given number. 

The functional (28) is calculated for 

rW=W 2 =Z'Za kj Wj + cZ 2 (34) 

Here the djcj and c are positive weighting coefficients. 

The optimal solution follows from eqns (30) and (31). The 
Liapunov function is a quadratic form 

(35) 

in which the Ajcj are real solutions of the system of algebraic 
equations 

a kj +2£ b ak A xj - L (£ m x AJ (£ m x A aJ ) =0 (36) 

satisfying Sylvester’s criterion. 
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The equation of the regulator is 

, I v 3k, +? f 
c=--l m *oir = z tor |(T|< U 

^ —c for a<-$ 

Here g is the linear function 

1 v 

V=--jLPc?1a 

P a = Z m fc(Aa + 4J 

which play the part of gain coefficients. 

Problem l is unsolved for c = 0. 

Optimization with a Non-stationary Functional. Problem 2 

With the aim of getting a transient process in Problem 1 
having more intensive damping of perturbed motions, one puts 
in (28) 


It has coefficients 


(37) 

(38) 

(39) 


rW = X(t)W 2 


(40) 


In this form the solution (27) remains valid, but now the 
eqns (43) cannot be satisfied by Ajcj = const., and one is faced 
with the problem of seeking its solutions for ffi[0,ao] that 
satisfy Sylvesters’s criterion. This problem is rather difficult 
and requires a special discussion. 

Non-stationary Linear Systems Defined over a Finite Interval. 
Problem 4 

If the bjc a and m k are defined for t £ [0, 7], Problem 4 may 
be reduced to Problem 3 by employing the transformation (14). 
A more direct path will be followed. 

Therefore consider the functional 


Here X (/) is an increasing function. For practical purposes it is 
convenient to take x(i)=*e 25t § > q (41) 

In this case the solution (37) retains its form. Only now the 
Liapunov function is y _ Q ibty^ (42) 

while the coefficients A k j are solutions of the Riccati equations 
A ki + 2 8 A kJ + fly + 2 Z - 

—-(EVJ(I m A) =0 ( 43 ) 


j= 


JL 5t 


The solutions are selected in accordance with Sylvester’s 
criterion. 

In case (41) one may put Ay = const. 

The point about the more intensive damping of the transient 
process is as follows: 

Since the characteristic index of X (t) is 2 <5, and the integral 
\oX(t) W 2 dt must converge, thus the characteristic index of r/* 
must always be strictly greater than d. Unfortunately for a reg¬ 
ulator designed in this manner the integral (28) becomes a mono¬ 
tonic increasing function of <5. This expresses the fact that one s 
wish to obtain a transient process with more intense damping 
of the oscillations cannot be matched harmonically to the con¬ 
dition for minimizing the energy consumption of the regulator, 
as expressed by the term c. 

Practically, satisfactory damping results may be achieved 
with sufficiently large S; the increments in the value of the 
functional will be insignificant provided the constant c is small 
enough. 

The above problem may be generalized within the framework 
of the same technique if the weighting coefficients are taken as 
given functions of time. 

Non-stationary Linear Systems. Problem 3 

One returns again to the linear systems (1), in which 

b kaL = b ka (t), m k = m k (t), te( 0,oo) (44) 

are known functions of time. 

The functional (28) will be optimized in which rW is defined 
by (40). 


W 2 dt + S Z z Cy (T) t]k (T) r,k 01 (45) 

Here S and T are fixed positive numbers, and the coefficients in 
IV% are known functions of time; the Ctj satisfy Sylvester s 
criterion. 

The optimal solution is provided by a Liapunov function of 
-form (42). As before, the coefficients of V. 2 satisfy the eqns (43). 
For these equations one has to solve Cauchy’s problem with 
t e [0, T]. The initial values of the variables are given for t = T, 
and in the integration time t varies backwards. 

In accordance with condition (43), the initial values are de¬ 
termined by the formulae 

A kJ (T) = SjC kj (T) ( 46 ) 

It is of interest to note that if eqn (17) exists, then it is continu¬ 
ously dependent on the parameter T, and in the case where the 
bjc a , mjc , djcj and c are constant at the limit as T -> oo, it tends 
to the equation appearing in Problem 1. This opens the way to 
the solution of Problem 1 on an electronic simulator. In fact, 
to obtain a solution of Problem 1 one must choose eqns (43) on 
an electronic simulator, inverting the time according to r = 

X _ t, and solve them for sufficiently large T with zero initial 

conditions. The values Aj C j = const, which provide solutions to 
Problem 1 are stable as r oo. Hence all solutions of eqns (43) 
that are found for A kj (7) = 0 will tend to A 1cj as t increases. 

Synthesis of Regulators Supplied from an Energy Source of 
Limited Power 

Regulators with Finite Actuator Velocity. Stationary Systems. 
Problem 5 

To return to Problem 1. Here one condition only will be 
changed: the optimizing functional. In fact, instead of the func¬ 
tion (34) consider the function 

rW= W 2 + cc£ 2 + C/; 2 : 


a>0 


(47) 


Application of the Bellman-Liapunov technique leads to the 
following equation for the regulator: 


0 for 

C°=Z for 


Zm« 

r 


>+£ 

<1 


(48) 


0 for 


The gain coefficients q a and r are found by the solution of 
equations similar to (43). One encounters them in Problem 6. 
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Recently A. I. Lur’ye has obtained interlocked formulae for 
computing p a , q a and r in terms of the parameters bjc ai m k and 
the weighting constants of the functional. 

Regulators with Limited Servomotor Velocity. Stationary Systems. 
Problem 6 

If in Problem 5 one requires definitely to make allowance 
for the fact that the servomotor velocity is limited and may only 
be chosen from the class B of functions /(cr), while at the same 
time there is no limitation on f. One may exercise this choice in 
the stationary case, keeping to the statement of the problem 
propounded on p. 251. 

Here one can not only determine the form of the servomotor 
characteristic f(o) s B precisely, but can also find the form of 
the switching function a (f] l9 ..., rj n , f, /). 

For the case w = 1, a = oo, and a quadratic functional of 
type (11) in which 

rW=W 2 +c c 2 + a<f 2 , a>0 (49) 

the servomotor equation is as follows 

+/ for a>+L 

h 

f 

f(a) = ha for \a\ <+- ( 50 ) 

-/ for a< —-L 

h 


In fact one puts in (28) 

rW=e 2St (W 2 + ce + ac 02 ) (55) 

The equation of the regulator retains the form (50), as the 
Liapunov function now appears as 

V^e 2 *% (56 ) 

Tts coefficients satisfy the Riccati differential equations: 

A kj + 2 5A kJ + a kJ + 21 b ak A XJ - L (£ £,.)(£ B s) = 0 

B k+Y b c,kBa+2 r £m OL A X j-L-(B k )=0 (57) 

R 2 + R 2 = oc(c + Y m k B k ) 

For stationary systems one can satisfy eqns (57) by putting A kh 

B k , and R = const. For <3 = 0 the solutions coincide with those 
of eqns (53). 

The solutions must be selected in accordance with Syl¬ 
vester’s criterion. 

General Case of Non-stationary Linear Systems. Problem 8 

Retaining all the assumptions of Problem 7, the optimizing 
functional only is changed. One puts 

/== J, e2 *‘( w 2 + c { 2 + oct 02 )dt + s['ZZC kJ (T) nk (T)r, J (T) + 


Here h is a given positive constant, while a is defined by 

a= ~2^hC B kn k +2RO (5i 

This solution is derived from the Liapunov function: 

K 3 = Z Z AjlkVj + C I! B k ri k + R£ 2 ( 5 : 

The coefficients V 2 are solutions of the equations 

%-=2Z ^-L(Z6 r )(ZB s )=0 

Z + 2 Z m cAa:j—— (B k ) = 0 (53 

R 2 =a(c+Y J m k B k ) 

V z being positive definite. 

Problems 5 and 6 may be solved by introducing the nev 
variable f and from there by adding to the model (7) the equa 
tion C = C one can synthesize the ideal regulator: 

t Oil.. iO ( 54 ; 

This approach may not be employed, however, if one wishes tc 
take account m the servomotor characteristic of its dependence 

mulae e (i°8H2 P 3T ameterS ’ “ ^ ^ that thiS W&S d ° ne in for ' 

For a = 0, Problem 6 is variable. 

Optimization with a Non-stationary Functional. Problem 7 

0 n, v °tL Can reP , e f‘ * e solution of Problem 6, changing in it 
only the form of the functional. 


+ £ (T) Z «j(T) >ij ( T ) + e (T) (T)] (58) 

Here 5 and T are fixed pbsitive numbers, and C, :j . dj and e are 
given numbers for which the corresponding form is positive 
definite. The optimal solution is provided by a Liapunov func¬ 
tion of form (56) whose coefficients satisfy the eqns (57). For 
these equations one has to solve Cauchy’s problem over the 
interval [0, 7], taking as initial conditions 

A kj (T) = sC kJ (T ) 

B k (T) = sd k (T ) ( 59 ) 

R(T)=se(T) 

Equations (57) may be used for solving the corresponding 
stationary problem, Problem 6 by the method described in 
Problem 4. 

Isoperimetric Conditions. Problem 9 

The basic problem may be complicated by isoperimetric 
conditions which represent some additional requirement placed 
on the system. 

For example, the limitation 

fr 

w ^+» £ 02 d t<l, m,n> 0, Z>0(60) 

Jo Jo 

represents the desire to achieve the best possible smoothing of 
the transient process under conditions where the actuator power 
consumption is strictly limited and so is its available displace¬ 
ment. In (34) and (40) one may put c, oc > 0. 

In other instances the isoperimetric conditions may concern 
limitations in characteristics of a more general nature, including 
the coordinates of the controlled plant. 
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In particular, for functional (11) with T previously specified, 
condition (10) may be considered as isoperimetric. Isoperimetric 
synthesis problems have scarcely been studied. Only for the 
simplest stationary case treated in Problem 1 has the following 
been established. 

Let it be assumed that in Problem 1 one has c = 0 in the 
function W 2 (34), while the limitation (60) with n = 0 is applied 
to the motion of the actuator. Then it has been shown that the 
solution of the isoperimetric problem does not differ from solu¬ 
tion (35) of Problem 1. Its special feature consists in the fact that 
the isoperimetric condition (60) leads to the appearance in the 
phase space of a closed region of initial values r\ 0 , containing 
the origin, for which the solution (34) exists. 

It has been shown that provided the initial perturbations are 
sufficiently small, the isoperimetric limitation is more powerful 
than limitation (33), i.e. for such perturbations the boundary 
in (33) is not reached for any t > 0. 


dependent of £, and that the following expansion in the coordinate 
7\ic exists in integral convergent series: 

>ik =Z +£ X c $ w? + • ■ • (63) 

a a p 

Returning to Problem 1, and retaining all the facets of the 
problem except that instead of initial linear equations one now 
considers the eqns (63). 

An optimal F-function may be sought as a sum of the forms 
F=F 2 + F 3 +F 4 + ... (64) 

expanded in integral values of the variables For example, 

(65) 

where the sum is calculated over all integral indices 
co — 1, ..., n with the condition 


Allowance for Perturbation Forces. Problem 10 

It may happen that the programme (5) will be determined 
approximately. This will be the consequence of either of the 
following reasons: 

(1) The solution (5) has been determined by numerical 
integration methods. 

(2) The eqns (1) take no account of the forces that are pres¬ 
ent in practice. 

In both cases eqns (7) will contain additional terms//^ (/). 

J7 = ify + mf+ /(0 (61) 

where/(/i, ...,/ M ) is a limited and vanishing vector function. 

In many cases these are known limited time-functions, 
vanishing at infinity, which represent determinate perturbing 
forces in (47). 

The presence of such functions gives one a basis for a state¬ 
ment of the synthesis problem for optimal regulators which takes 
into account the known information on the forces f(t) and 
improves on eqn (12) as previously found without making 
allowance for these forces. 

The Liapunov function has the form 

v = ZZ A kj f h>1j + ZA(0^ + £(0 (62) 

Here the first term coincides with V 1 ; the values of Bj c ( t ) and 
D (t) are determined from the known fk (t), and they are vanish¬ 
ing functions provided the fk (t) are vanishing. 

The sign of V is determined by its first term. The equation 
of the regulator splits up into two parts: in the first it coincides 
with eqn (37); while in the second it contains terms dependent 
on the fk (t). The regulator operates on the displacement prin¬ 
ciple and the perturbation principle. It achieves a lower value 
of the optimizing functional than would be obtained if informa¬ 
tion on the perturbation forces were not made use of. The very 
simple problem considered is capable of effective generalization 
to more complex cases. 

Effect of Slight Non-linearities . Problem 11 

In a number of applied problems it is extremely desirable to 
allow for the effect produced by non-linear terms (V/, £„ t). 

Now study the treatment of the simplest case. One assumes 
that eqns (7) are stationary, that the non-linear terms are in¬ 


a-b^+ ... +co = m 


( 66 ) 


In attempting to satisfy eqns (30) and (31) with this form, one 
arrives at systems of recurrence equations for the coefficients 

A m 

a, <D* 

Thus one obtains eqns (36) for the form V 2 , which agrees 
precisely with form (35). It is a characteristic feature that if 
eqns (36) are soluble, then the systems of recurrence equations 
for determining the coefficients of the other forms are also 
soluble. 

As before, the equation for the optimal regulator is deter¬ 
mined from the formula 




+ f± y 

-1 c Lmk dr, k 


= 0 *, 


C7>+j 

m< i 
0-<-£ 


(67) 


wnose right-hand side now consists of an infinite series of integral 
powers of the coordinates ??&. The convergence of the series (64) 
and (67) is ensured, at least for sufficiently small \ \r]\\. 

No difficulty arises in principle for applying the technique 
also to those cases where the right-hand sides of eqns (7) can be 
represented as integral series of a small parameter e. The same 
rule holds good in all cases: the non-linear problem is soluble 
provided the linear corresponding problem is also soluble. 
Here lies the great value of investigating the synthesis problem 
for linear systems optimized by a quadratic functional. 


Effect of Slight Non-linearities. Non-stationary Systems. 

Problem 12 

The assumption of the stationary nature of system (63) can 
be omitted from Problem 11. Eqns (30) and (31) may be satisfied 
by the function K(64) if one assumes that its coefficients A™ & 

are dependent on t. There will be systems of recurrence Riccati 
equations to determine them. 

As before, the equation of the regulator is determined from 
the formula (67): but now the coefficients of the series will de¬ 
pend on t. As for the properties of the Riccati equations, one 
may draw a number of conclusions in this case which are similar 
to those derived in Problem 4. 


More Complex Criteria Differing from the Quadratic. Problem 13 

It must unfortunately be stated that no instance is known of 
the effective solution of synthesis problems in which the optimiz- 
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ing criterion is other than quadratic. Nevertheless, one problem 
is indicated here that is entirely convenient for systematic study 
by the techniques described above. The case in mind is that of 
an optimization criterion chosen such that the functions W and co 
may be represented by integral series of rj, f and |°. The coeffi¬ 
cients of the series may be time-dependent. In these cases one 
can develop the method of solution set out in problems 11 and 12, 
even if the controlled plant is non-stationary and contains non¬ 
linear terms 3 (rj, /). 

Regulator with Delay. Problem 14 

In the following substantial development of the ideas of 
pp. 251—252 and Problem 6 for taking account of possible 
characteristics of the technical equipment employed, using the 
concept of incomplete observability, a most effective result has 
recently been obtained on the basis of the following assumptions: 

(1) In the control process it is only possible to measure and 
store, during a limited time, the quantities W l9 ..., Wi(l < n), 
which are related to ?? by the vector equation 

W=<f>(t,ri), (/>{(!> ( 68 ) 

which is not uniquely soluble for rj. 

(2) The measurement of the components of the vector W 
occurs with a time delay 0. 

(3) The time T = co. 

Then the equation of the servomotor takes the form: 

C° = /l>(f + 0), £0 + 0)0 (69) 

— t <0 <0; 0 < t = const. 


Consider now the functional 
r 

o + (73) 

It is defined for the class G of continuous-segment £ and 
smoothed-segment functions ')]. Obviously the control must 
depend on the state of the system at the instant t and also on 
its state over the preceding time interval [t + 0, t] 9 — r < 0 < 0. 
Hence the equation of the optimal regulator, if any, will be of 
the form: 

F K (0, n (f), rj(t + 6X t]= 0 (74) 

-T<0<O 

The following problem arises. For the class G of permissible 
comparison functions, it is required to determine a regulator 
eqn (74) such that: 

(1) it minimizes the functional (73) for all motions of the 
system starting from the set M (72). 

(2) it makes the closed system (71), (74) stable. 

The equations of pp. 250-252 are capable of generalization 
to systems of type (71), (74), and the regular procedure of 
applying them may be developed for all the particular cases of 
the problems described above. 

Thus, for example, in the case of minimizing the quadratic 
functional W 2 (34) in its application to a linear stationary con¬ 
trolled plant 

'ft (0 = I (btJIa + Ckaflx (t + 0)) + m k Q (t) ( 75) 


In these assumptions one is seeking the precise form of the func¬ 
tions ‘/’ appearing in eqn (69), such tha t: 

(1) The unperturbed motion of the closed system (7), (69) 
should be asymptotically stable. 

(2) For all perturbed motions arising from (9) and £ (0) the 
functional (11) should be a minimum, the W in it being a positive 
definite analytic function of the variables rj, £ and £°. 

In particular, in the linear case, the required control law 
has the form: 


C = r(t)f + 


[N(t,9)W+M(t,d)s(t + Q)]d9 (70) 


Here the functions r, N and M are continuous in t and 9 and 
uniformly bounded with respect to t. 

A computing algorithm has been developed for determining 
them, which, in particular, includes a solution of the Riccati 
equations. 


More General Case of Systems with Delay. Problem 15 

Let equations of perturbed motion be considered under the 
form 0 

r lk=fk \jl OX n 0- t), & t] (71) 

defined in a certain region N (rj , |) > 0 and t e [0, T]. 

One assumes that the motions are determined for a certain 
initial set of functions M 

nif)=g(t) ( 72 ) 

— T<t<0 


~~T<0<0 


the problem is solved by a Liapunov function of the form 






Pki(t,9)rii(t+0)d0 


+ 


ofo~ 

ZE vkiO, 

“t J ~r L k i 


0 ? h) rj k ( t + 9) }] k ( t + h) d 0 


dh (76) 


The equation for the optimal regulator is: 


— Ew/*(0+E 

C k k 


(t,8)ti k (t+6)d0 (77) 


Here the pi- are constants or functions of t, while the c/k are 
functions of t and 6. Computation techniques have been de- 
veloped for determining them. 

It is appropriate here to mention an approximate method for 
solving the problem due to M. E. Salukvadze, based on an 
application of the Lagrange finite-increment formulae. The 
essence of the method is as follows. The delay interval [t — /?, t ] 
is split up into m equal sections, over which the function is 
described by means of the Lagrange formulae. 

The formal introduction of new variables allows one to put 
this representation into the form of a system of linear differential 
equations. Then the original problem is completely replaced by 
another problem in which delay plays no part. Its solution is 
found by the methods described earlier. The value of the solution 
lies in its rapid approach to the accurate solution, as for example 
(76), (77), as the number m increases. 
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Terminal Control . Problem 16 

Problem 16 follows from the basic problem pp. 247-248, if 
r ~ 0 is put in (11). The system is optimized according to its 
terminal state at the instant t = T. Although this problem fits 
within the general framework of the methods described here, 
it has its own specific features and has been little developed. 

The formulation of the problem is as follows (linear case). 
Consider the perturbed motions described by the linear 
eqns (7). Here B (t) and m (t) are continuous matrices, defined 
for t & [0, r], while the f are continuous-segment functions 
satisfying the limitation (33). 

Out of the class of permissible functions it is required to 
determine an eqn(17) for the regulator, for which the functional 
(11) attains a minimum for all motions starting at (9). 

The problem calls for an application of Pontryagin’s maxi¬ 
mum principle method, which allows one to write down all the 
necessary relations for a solution. The control law is: 

£ = £ sign (mV) ( 78 ) 

where the vector ?/> satisfies the equation 

t//=-£ 5 V (79) 

The asterisk denotes transposition of matrices. Success in 

applying the maximum principle is ensured by the fact that one 

can write down the following boundary condition for the vec- 

tor?: .KT)=-grad w07(T),T) (80) 

Consider now the space A of initial t] 0 , and the space B of 
terminal states rj ( T ) of the controlled plant. 

The equations for the problem enable one to set up a one- 
to-one correspondence between A and B. They also allow one 
to determine the instants for the switchings of the 

actuator, the number of these, and the points r] (ti ) correspond¬ 
ing to them in the space of states 77 . 

Effective calculation of these relations can only be carried 
out using a computer, even in the simplest stationary cases. 

Regulators Optimized for Response Speed. Problem 17 

The following is a statement of the simplest problem in the 
synthesis of regulators optimized for response speed. 

Consider a stationary linear system (7), defined for t e [0, 00 ]. 
Let 0 < T < x , and determine the point 

rj(T) = 0 (81) 


process in a finite time, one may assume in advance that the 
class G of permissible functions consists of continuous-segment 
£ and smoothed-segment rj. The eqns (30) and (31) for the prob¬ 
lem are 

=1+1(1^+'^)^- ( 83 > 

k a 


a t 




. 7 . v W 
C = C sign L ,? ifc ^ 


(84) 


It would seem that if one were to solve eqns (83) and (84) and 
find the Liapunov function V, then the equation 


S-SS" 0 (85, 

would define the law for switching f. Here lies a difficulty, 
however, which one does not know how to overcome. In fact the 
representation (84) contains two equations, which must be satis¬ 
fied by two functions V x and V. 2 : that is, the equation 


QV 1 . 0F, V dV 




and also the equation 


dV 2 , QV T V 31 7 


( 86 ) 

(87) 

( 88 ) 


V |L< 0 (89) 

ZjW,c drj k 

Obviously the functions V x and V 2 must assume identical values 
(by virtue of the continuity of V) in 



A solution of the problem has not yet been achieved along these 
lines, and it is not known how to do it. To the engineer who is 
able to find a known answer in an approximate equation for a 
regulator optimized for response speed, one can only recommend 
the expedient of replacing the original precise problem by an 
approximate one. 

One such expedient has been indicated in Problem 2; another 
consists in optimizing the functional 


If there exists a regulator (17) such that: 

(1) it satisfies condition (33) \£\ < £; 

(2) when connected to the plant (7) it damps out any motion 
starting at (9), i.e. the solution 

rj(o o) = 0 (82) 

is asymptotically stable; 

(3) it brings the system to the position (81) in a minimum 
time T: 

then one can call such a regulator optimized for response speed. 

The problem consists in determining eqn (17) of such a 
regulator from a class of suitably defined functions G. It falls 
within the outlines of the basic problem if in (11) one puts 
W== 1 and co = 0. Its special feature is as follows. Since it is 
required in observing condition (33) to complete the transient 


J = 


LtL — )-rv IT 


Here 0 < 0 < 1; for 0 = 0 one gets Problem 1, while for 0 = 1 
one has an approximate version of the response-speed problem. 
The larger 1 and the better is this version. 


Random Processes 

The range of synthesis problems broadens considerably if 
the model (7) [(61)] contains elements that vary randomlv. 

Such elements may be: 

(1) the programmed motion, 

(2) the parameters of the model, 

(3) the acting perturbation forces fit). 
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In another case one may wish to make allowance for the 
technical imperfection of the control equipment and to include 
the noise it generates in the synthesis problem. 

If the probability characteristics of such random processes 
are known, the synthesis problem is formulated in terms of the 
theory of random functions. Essentially this problem has been 
conceived in the works of Kolmogorov, Khinchin and Wiener. 
They have laid the fundations of the modern theory of filtering. 
This theory is a large and independent section of the theory of 
automatic control, and has an extensive literature. The problem 
of synthesis of optimal systems with random properties may also 
be treated from the standpoint adopted in the present paper. 

In fact, in a number of cases the use of the terminology of 
random function theory enables one to reduce the synthesis 
problem to a normal variational problem, into which one trans¬ 
fers the Bellman-Liapunov method of solution described here. 
This is demonstrated in the following example. 

Let the equations 

^=/(^,c\*0+y(0 (92) 

describe a random process (perturbed motion) starting at (9). 
Here : f], f and t have their previous meaning; x (t) is a limited 
Markov process representing the variation of a certain para¬ 
meter, and y is a noise. Their distribution functions are given. 
For the sake of simplicity we shall consider that y -> 0 if x -> 0. 
We assume that the equations are defined for \\r)\\< R,te [t 0) co], 
R being a given number. We introduce a definition of stochastic 
stability. 

The motion rj == 0 is called probabilistically stable if for any 
two numbers s > 0, p > 0 there exists a d > 0 such that the 
inequality 

P {Ik (Oil <£ is satisfied for all f>0/||»/(0)|| <8} 1-p (93) 

Here P ( cp\xp ) is the conditional probability of the event <p under 
the condition y>. Furthermore, if the condition 


Problem: for permissible functions find an equation 

i>t,x) 

of a regulator such that for £ = f 0 : 

(1) the functional J is bounded; 

( 2 ) a minimum w.r.t. f is attained for any ?] 0 from ||?y ( 0 )|| 
< R 0 and any x u ; 

(3) The solution rj = 0 is probabilistically stable; 

(4) p {IMI < R for all t > t 0 | II rj ( 0 )|| ^ i? 0 } > 1 ~q for 
any rj (0) from | \ij (0)|| < P 0 and any a: (f). 

To solve the problem, we employ the Bellman-Liapunov 
method in the following version. 

If there exists in the region | \y}\ | < R a function v (?/, x, t) and 
a function f° ( 77 , x, t) such that: 

( 1 ) v is defined in ||?y|| < R ; 

( 2 ) v is positive definite; 

(3) at the point rj = 0 there exists an infinitesimally small 
upper bound; 

(4) Sup [Vnpu\\rj\\ < R 0 ] < inf [Vnpu ||^|| = R] q 

(5) the averaged derivative dM {v]/dt 9 evaluated along the 
trajectories of the closed system, satisfies the equation 

“W = _^ 

dr 

and the optimization condition 


d M{v} 
dr 


ICO 


-f 


d M{v} 
dt~~ 


+ W 


= 0 


(6) v (r, x) = r (r, 7], x ) on the surface ||r/|| = R: then £° 
is an optimal control, and 

v = min J 
i 


[lim P {||/j(0li <cp} = 1 

f-+ CO 

is satisfied for any a> > 0, then the stability is said to be 
asymptotic. 

Let a positive function W (>/, >, /), definite in ?/ and £, and a 
positive definite function r ( t, rj) be given. Let the region of 
values of rj that are permissible in the control process be defined 
by the inequality 

I M\<R (93a) 

for all initial values from 

||»7(0)|| <R 0 ,R 0 <R (93b) 

and we have to ensure that condition (93 a) is satisfied with a 
given probability 1 — q. Here 0 < q < 1 (a small q is desirable). 

We shall characterize the response of the transient process 
by the conditional mathematical expectation 

W (if, x) d r + r ( T ; n ( T))J | n 0 , x 0 j 

lkoll<-R 0 ^-R (94) 

Here T is the random instant at which the achievement of the 
trajectory first attains the surface ||^|| = R 0 . 


Jblo 9 Xo>to ) = M 


The function V (?/, x, r) which satisfies these conditions is called 
an optimal Liapunov function. 

There also exists a variant of the Bellman-Liapunov method 
for the case in which equations (2.6) are of more general form 
and are defined for t e [/ 0 , 7]. 

This method enables an effective solution to be obtained 
in all the cases of given initial equations, optimizing functionals 
and characteristics of random processes that have been de¬ 
scribed above. Many of these results have already been derived. 
Both deterministic and stochastic processes admit optimization 
by the method described in many cases where they are of a 
discrete nature. 

Processes with Adaptation 

Methods have been outlined in the literature for the study 
of processes with adaptation. They are partly based on the Bell¬ 
man optimization principle and the methods of probability 
theory. The question whether it is possible to develop a technique 
for studying processes with adaptation similar to that employed 
here cannot be answered now. One half of the method exists. 

The immediate future will show whether an adequate second 
half of the method—some variant of Liapunov (or perhaps non- 
Liapunov) stability—will be evolved. 
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Mathematical Aspects of the Synthesis Problem 
Existence of Solutions 

The account in pp. 250-254 probably exhausts all the known 
particular synthesis problems that have been solved or can be 
effectively solved. 

In this problem of the existence of the solutions themselves 
has not been considered. This extremely important, complex 
and interesting problem has been substantially developed in the 
theory of the maximum principle, in a number of papers by 
Bellman, Filippov, Kirillov, Kulikowski, and other authors. 

For this purpose one could limit oneself entirely to an ex¬ 
position of Kalman’s highly elegant theory of the controllability 
of linear systems. To a considerable extent this theory is devoted 
to the solution of the existence problem. From the point of view 
adopted, however, in this paper, Krasovskiy’s theory of stabiliz- 
ability is more acceptable. In all publications the existence prob¬ 
lem is reduced to the solution of two questions. 

It is assumed that a permissible control exists. Then does the 
existence of an optimal control follow from this? The answer 
appears to be affirmative for a broad class of cases. Given 
this condition, the second question is: do permissible con¬ 
trols exist, and if so in what region N (.x , p) > 0 of the phase- 
space? The theory of stabilizability studies the eqns (7) in the 
neighbourhood of the origin. Let it be assumed that the equa¬ 
tions do not depend explicitly on t and that the optimizing func¬ 
tional is quadratic. 

Consider the vector subspaces corresponding to the eigen¬ 
values 2 of the matrix B — const.; 

(1) The subspace S + corresponding to the eigenvalues 2 for 
which Re 2 > 0; 

(2) The subspace S° corresponding to the eigenvalues 2 for 
which Re 2 = 0; 

(3) The subspace corresponding to the eigenvalues 2 for 
which Re 2 < 0; 

and the space R consisting of the vectors 
m, Bm ,..., B n ~ l m 

Then the following general result is obtained: 

A. The system (7) is stabilizable no matter what the form of 
the non-linear terms E (rj, t), provided the enclosure condition 
S + + S° s R is satisfied. The law of the control £ is a linear 
function of the type £ = %pp]j. 

B. The system (7) is not stabilizable by means of a linear 
control law no matter what the form of the non-linear terms 
3 (rj, t ), if the enclosure condition is not satisfied and there exists 
at least one vector Sj s S~ h that does not belong to R . 

C. One is obviously dealing with a critical case if the enclos¬ 

ure condition S + e R is fulfilled but there is at least one vector 
S] s 5° that does not belong to R. In every critical case the 
question of the stabilizability of system (7) by the control 
£ = must be solved with reference to the nature of the 

non-linear forces. 

This theory is capable of generalization to non-stationary 
systems, systems with delay, and non-quadratic functionals. 

Continuity of the Liapunov Function 

The use of the Bellman-Liapunov method requires an ex¬ 
amination of the fact that the function V is continuous in every 
problem. One returns now to Problem 1. 


Here the region N ( x , ju) > 0 is divided into three parts. 
For example, when \£\ < £ the function V is defined by formula 
(35). Since V is positive definite, it represents a paraboloid in 


the 7], V space. When £ = ± £, the paraboloid is intersected by 
the planes 

<«) 


which are parallel to the V axis. 

The geometrical locus of the intersection points is described 
by a (n — l)-dimensional form. 

Let it be assumed that p n 4= 0 (37). 

From (95) is found 





c(±0~ 


n -1 

Z ?A 


(96) 


This form is of the type: 

(97) 


The new variable is introduced 


EpA+c(+0=( ' (98) 


and reverts once more to the optimization equation. 
If the system is in such a position that 



07 

d>1k 




the actuator is on the stops ± £, and the equation for determin¬ 
ing V becomes linear in the partial derivatives. 

The coordinate r\ n is eliminated and another linear equation 
is obtained in the partial derivatives: 


1 (99) 

where Xjc and X depend only on %, ..., rj n - l9 and f. 

Thus one has arrived at the following Cauchy problem: it is 
required to find a solution V (rj l9 ..., r} n - Xi '£) of eqn (99) which 
for £ = 0 assumes the value V 2 determined by formula (97). 

Such a solution exists, at least in a certain neighbourhood 
containing the origin. 

In the general case this problem remains open to investi¬ 
gation. Where there are discontinuities in the function V, the 
synthesis problem requires another statement taking into ac¬ 
count the steps A 7 on the lines of discontinuity. 


Conclusion 

The author has tried to show that: 

(1) The problem of synthesis of optimal regulators is funda¬ 
mental to modern automatic control theory. 

(2) Its rigorous statement is based on Liapunov’s concept of 
perturbed and unperturbed motions. 

(3) The physical and mathematical content of the problem 
is determined by the nature of the controlled plant and by the 
form of the optimization criterion. 

(4) In many cases of deterministic and stochastic systems 
which are of practical interest in their own right, an effective 
solution of the synthesis problem has been and may be achieved 
by using both the methods of the calculus of variations and the 
Bellman-Liapunov method. 
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(5) In these cases it is of fundamental importance to solve 
the simplest problem on the optimization of a linear system by 
means of a quadratic functional. 

(6) Effective criteria are now available for telling whether a 
synthesis problem is soluble or not. 

(7) A persistent search can be seen in the literature for new 
methods of solution—functional analysis methods and approxi¬ 
mate methods—and also for new cases where the problem is 
soluble (in the field of continuous, discrete, and stochastic 
systems), so that a clear outline is beginning to emerge of a well- 
built theory based on a unified method. 

(8) The subject matter of investigations into the synthesis 
problem has good prospects of development in the next few 
years. 

Critique of Modern Automatic Control Theory 

A Particular Question in the Statement of the Optimal Control 
Problem 

H a PP ea rs that by far the greater part of publications on 
optimal control theory is devoted to the problem of choosing 
brachistochronic motions (5). This is natural. It means that the 
boundaries of the region N (.x , p) > 0 are chosen to be extremely 
wide. This choice is not justified, and in a number of cases may 
make complete nonsense of the optimal control problem. Atten¬ 
tion is drawn here to the existence of a deep connection be¬ 
tween the problem of choosing a control programme and the 
basic problem. This connection exists independently of whether 
the programme (5) is a solution of some variational problem or 
whether it is any other manifestation of its creator’s will. The 
connection is two-sided. 

First, it is stressed once more that since the model (7) is de¬ 
fined for the motions (5), there is no a priori reason why it should 
be completely controllable or stabilizable. 

Secondly, in all cases where the initial equations (1) are de¬ 
fined in a closed region N (x, p)> 0 the programmed motion (5) 
must not cross the boundaries of this region. 

By using the simplest example the terminal control problem 
is now illustrated. It is assumed that the closed region is a 
layer of infinite extent and thickness 2 ju defined by the inequality 
\lA < pi. Let p* be some control programme. Then 

-p<p* + %<p ( 100 ) 

Obviously from this it would be rather unreasonable to choose 
the programme (5) as follows: 

/** = /sign/(f) (101) 

Here/(0 is some switching function. If the number 2 ju actually 
exhausts the whole range of actuator displacement, then one is 
deliberately imposing very severe conditions on the actuator. 
Its displacements must satisfy the inequality 

/i(0<£</ 2 (0 (102) 

where / x and / 2 are given functions of t which assume values 
identically zero over whole sectors. Then it is always possible 
in the space A to designate some region A z of initial values rj 0i 
adjacent to the origin, which possesses the property: any sector 
*0 = I tool I of transforms into a sector R t = \ \y]t\\ of space 
B such that Rt > R Q . The greater T, the more intense is this 


inequality. Obviously with the programme (101) the optimization 
problem loses its meaning. This is splendidly illustrated by the 
following example. 

Let it be assumed that one is controlling a car on an abso¬ 
lutely rough horizontal plane. 

The position of the car is defined by the coordinates x, y of 
its centre of gravity, the angle yj between its plane of symmetry 
and the y-axis, and the angle p of rotation of its front wheels; 
the positive sense of the angles is taken as counterclockwise. 

Suppose that it is required to choose a programme for optimal 
control of the car under the following boundary conditions: for 
t = 0, the coordinates of the car x 0 = y 0 = 0, yj Q = 0; while for 
t — T one has xt < 0, yx > 0, and yj = yjT > 0. 

The solution is as follows: the wheels are rotated through an 
angle t u* = ju, which will ensure that the point xt, yr, y>T is 
reached. There is no requirement for switching of p. 

Now suppose that the car has been replaced in its initial 
position x 0 — y 0 — 0, but with an angular error A yj (0) < 0. 
Obviously in this case the point xt, yT will never be reached. What 
is more, however small Ay; (0) is, with the chosen control pro¬ 
gramme p* = p one can never be able to reduce the terminal 
control errors, which will be monotonically increasing with time. 

Thus in formulating a statement of the optimal control prob¬ 
lem one comes up against a problem of compromise in the 
separation of ‘spheres of influence’ such that allowance should 
be made both for the interests of control programme selection 
and those of optimal regulator design. 

In many publications the subject is discussed in such a way 
as if this problem did not exist. 

Critique of the General Approach to Optimization. From Letters 
to the Author 

Letters to the author of this paper from Professors Balchen 
and Zadeh have drawn attention to the narrowness of the en¬ 
deavour to optimize a system according to a single scalar 
criterion of the type (11). 

In fact, when considering various vital situations in the broad 
sense of the word one intuitively tended to choose a plan of 
action under given conditions by assessing the situation from 
various points of view, which cannot be expressed by a common 
scalar criterion. Hence the idea is advanced in these letters of 
evaluating the response of a system according to a vector 
criterion. The author is in agreement with this criticism, but it 
is not possible here and now to give a precise statement to the 
problem of optimizing a system according to a vector criterion. 
This problem will form the subject of investigations in the im¬ 
mediate future. 

Here only the speculation that it may perhaps not prove pos¬ 
sible to formulate this problem correctly within the framework 
of the model (7) has been made. For this purpose it will possibly 
be necessary to consider as controlled plants what is now 
normally known as ‘large systems’. 
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Approximation Methods in Optimal and Adaptive Control 

J. H. WESTCOTT, J. J. FLORENTIN and J. D. PEARSON 


Summary 

The mathematical equations for optimal and adaptive control are 
now known. However, comprehensive methods for their numerical 
solution are not available. This paper gives a discussion of approxi¬ 
mate computer methods of finding numerical solutions to suitable 
engineering accuracy. The importance of setting up the problem in a 
suitable form is stressed, and illustrated with an example. Three 
explicit approximation techniques are described, one a trajectory 
method, and the others function space methods. For each technique a 
worked example is given. 

Sommaire 

Les equations mathematiques des systemes d’optimalisation et 
d’adaptation automatique sont connues. Ce qui Test moins, ce 
sont les mSthodes numeriques permettant d’en determiner les solu¬ 
tions. Le rapport decrit quelques methodes d’approximation au 
moyen de calculateurs conduisant a des solutions numeriques dont 
Fexactitude est suffisante pour les besoins de la pratique. II met en 
evidence la necessite de mettre les donnees du probleme sous une 
forme appropriee et illustre cette necessite par un exemple. Trois 
methodes d’approximation explicites sont decrites, l’une se rapportant a 
une trajectoire a une dimension, les deux autres a des fonctions a 
plusieurs dimensions; chacune de ces methodes est illustree par un 
exemple., 


Approximation may be attempted either in the setting up of 
a particular problem or during the numerical process of solution. 
Whilst approximations in the setting up procedure are some¬ 
what difficult to treat analytically they are likely to be important 
in practical applications of decision theory, and a worked 
example is presented to demonstrate some aspects of this. In 
numerical techniques two broad approaches can be distin¬ 
guished—trajectory and function space methods. Certain ap¬ 
proximation techniques in both these classes are discussed and 
illustrated with simple examples. 

Review of Basic Equations 

To introduce the approach and notation, a brief review of the 
basic equations will be given. More complete derivations can be 
found 1 - 2 . The complete system including inputs must first be 
described by a set of state coordinates x(t). These are an absolute 
description of the system at the time instant t. The state co¬ 
ordinates may be obvious physical quantities such as position 
or velocity, or may be statistical quantities such as the mean 
and the variance. The motion of the system is conveniently 
described by a set of first order vector differential equations: 


Zusammenfassung 

Mathematische Gleichungen fur optimale und selbsteinstellende 
Regelungssysteme sind bekannt, jedoch gibt es noch keine umfassen- 
den Methoden fur deren numerische Losung. Dieser Aufsatz behandelt 
einige auf dem Rechner durchfuhrbare Naherungsverfahren, die zu 
numerischen Losungen fuhren, die fur praktische Zwecke ausreichen. 
Besonders wird darauf hingewiesen — und anhand eines Beispieles 
erklart — wie wichtig es ist, das Problem richtig zu formulieren. Drei 
explizite Naherungsverfahren werden besprochen, von denen eines auf 
der Darstellung durch Trajektorien und die anderen auf der Dar- 
stellung im Funktionenraum beruhen. Fur jede dieser Methoden wird 
ein durchgerechnetes Beispiel angefuhrt. 


Introduction 

Both optimal and adaptive control problems can now be tieated 
by the same decision theory approach 1 . Typical practical prob¬ 
lems can be formulated in the required mathematical terms, but 
at present there is still difficulty in determining actual numerical 
solutions to problems of realistic size and complexity. It seems 
likely that a variety of approximate computation techniques will 
be developed, each with a restricted range of application. 

Approximation is necessary due to the very extensive cal¬ 
culation called for, using multi-stage decision methods. These 
become sufficiently time-consuming, even when performed at 
the fast speeds of modern digital computers, for abbreviation to 
be necessary. Approximation becomes attractive due to the 
limited time allowable on the time scale of the dynamic process 
it is desired to control. 


x' = A(x, a, t) 


( 1 ) 


where u is a vector of control variables. For reasons of simplicity 
no random components are included. 

An optimal control function u(t) is to be found which 
maximizes or minimizes a given performance index, subject to 
constraints. Typically the performance index will be a path 
integral over a defined time interval t, T with the system starting 
at a given position x. Here the performance index will be 
defined as rx 

f(x(t), t)= min L (x, y, A) dA (2) 

U J t 

Isolating a small part of the time interval t, T, it can be seen 8 
that the following problem is equivalent to (2) 


fix if), t) = min 


ft + A 


L(x,u,X)dX+f(x(t + A),t + A) 


( 3 ) 


Using a Taylor series expansion for the third term, a partial 
differential equation for the performance index can be found in 
the form 

^-=—min \iA^+L (4) 

8f a Li dX i J 


The solution of this partial differential equation reveals the per¬ 
formance index as a function of the initial position and time 
interval. The minimizing control law u(t) can be found as a 
function of the partial derivatives of /. 

With appropriate modifications analogous equations can be 
developed for discrete time systems and for those containing 
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Wh ! n th6re are random Aments the per- 

pathJLntegrals** *° “ »er,ge of 

aItefn 0 a r tii t ^ m,n i StiC ; T*™ ^ (3) Can be expressed in an 
don /Z al y ^ ltS Characteristics - Define the func- 


x~~ 


he set of equations for the characteristics is then 
Sff . QH 

dp p Q x (6) 

Si” r„sr pond d,her ,o * ^ e » d »»“• 

x(T) known p(T) unknown ( 7 ) 

or to a free end point variational problem in which 

x(T) may be chosen p(T)= 0 (8) 

value one. CaSC ^ Pr ° blem reduces t0 a two-point boundary 

Computational Methods 

techn^QuefVh^i 6 ^'^/' 36 ^ 0 tW ° families of computational 
- v ._ t u ’ Th first IS of Wlde application, it computes f(x t ) 
over the x space at successive time intervals t r . It will be termed 

t be fanCtl0n space me thod. A predominant difficulty in the func- 

t on n a affip-tT aCh * ° f St ° ring a “^i-dimensional ffinc- 
tion in a digital computer. The obvious methods become pro- 

pohitslnearh ? * three ' dimensi onal function with a hundred 
necessary dlmensi °n a storage space of 10 6 words would be 

is tJrmed^Z 1 appHeS 0nly t0 deter ministic systems and 

' * e trajectory method. Only half of the boundary 

Setting Up Problems 

In practical situations the problem of control is usually not 
completely defined. In many cases it is possible to complete the 
specification by the selection of constraints which enable a 


simpler solution to be achieved than would otherwise be pos¬ 
sible The factors which can most usefully and easily be varied 

Ze^cT f ° rm ° f the performance index, the precise 
specification °^. the contro1 constraints, and the selection of 
continuous or discrete time working. 

To illustrate the effects of varying the formulation consider 

r;:r r pr °“ em - a vehicie > m ° ving with 

velocity v in the x v x 2 phase plane, is to be guided from an 
initial point P to a final target O. Guidance is affected by ad- 

maxTmum ° f the Vehide ' For P ractical reasons the 

Fi^nu'T ° f - n “ constrained - In accordance with 
Hgure 1 the dynamic equations of the problem are: 


x[ =vcosx 3 
^2 = t;sin x 3 
x'->=u 


( 9 ) 


where x 3 is the angle the velocity vector makes with the .v x axis. 

Minimum Time Trajectories 

Take the performance index to be 


/ (x 5 zO = min 


h-dt 


Si - , d :Se« 4 “ ° * re8i °" * h ‘ 

Following the usual formulation 

H =min [h+ Pl v cos x 3 +p 2 v sin x 3 + p 3 «] 
leading to the optimal u being 

u = — sign (p 3 ) \ u \<2 

S SSSSf” fo,low from ,te ‘“sues „f *• 


Pl=P2=0 

P , 3 : =PiVsinx 3 -p 2 vcosx 3 


( 10 ) 



\- / 

dtpeSffieont?^ 68 T COmposed of stra ight lines and circles 
depending on the boundary conditions of Pl and p. 

As a physical consideration the additional definition of 
sign(0)=0 

minimizes the number of switchings in u and the control is one 

ffieT^tT^ ‘ bang - bang ’ ^ chosen initial poLt P 
the optimal trajectories are at A in Figures 2 and 3. The vehicle 
reached the origin 0 in 3-24 sec. 

Minimum Energy Trajectories 

If the performance index is taken to be 


/ (x, t) 


= min 

v LJ t 


h+~) dl 
2v 


wiffi V as a yet unchosen Lagrange multiplier, the optimum « 
s 8 7 7 The trajectories equations are unchanged At the 
terminal point 0, the x 3 boundary is unspecified and consequently 
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Time, sec 



p z and u are zero. To the first order of approximations along 
the trajectory through the origin 


t „ dxi dx 2 . 

dr=—- cos x 3 4- 1 sin an 

J V i V 5 

and hence 


0/ _ _cosx 3 0/ __ 
dx t v dx 2 P 2 


sin x 3 
v 


The problem of solving the trajectory equations reduces to the 
selection of a terminal x 3 and adjusting it until the initial 
boundary values are satisfied. For the given values of v — 3 the 
optimal trajectories are shown at B in Figures 2 and 3. The 
system reached the origin from P in 3*27 sec. 


Minimum Squared Miss Distance 


The performance index is taken to be 


f{x, 0 = min 

U 




A(T) + 


_1 

2 


4(T) 


resulting in the same control signal a — — v p 3 . The trajectory 
equations are those of (10) and (9) and the boundary conditions 
are clearly p x ( T ) = x l9 p 2 ( T ) = x 2 , p 3 (T) — 0. Since the target 
at 0 is the origin these values closely approximate those for the 
minimum energy trajectory presented earlier in this paper. The 
optimal solutions are those for C in Figures 2 and 3 obtained 
for v — 100, and the minimum time achieved was 3*27 sec. 


Summary of Formulations 

Comparing the energy used in each of the four schemes 
reveals that apart from the minimum time trajectory there was 
little difference. The minimum time control scheme used 2T6 
units as opposed to a minimum energy formulation using 
1*23 units. 

The insensitivity of the solution to the formulation suggests 
that the performance index producing the most tractable set 
of equations should be employed. In this example the minimum 
time solution is a geometric exercise, whilst the formulation of 
the minimum squared deviation and control energy generated 
a rather troublesome set of equations from the numerical point 
of view. Clearly, in general, there is room for experiment within 
the constraints of the problem. 


Approximation Method for Boundary Value Problems 

Since the trajectory methods lead to two point boundary 
value problems, methods for dealing with these are receiving 
a great deal of attention. Boundary value problems are far older 
than numerical variational methods, and many well established 
techniques exist in numerical practice. It is essential to distin¬ 
guish between the time available for calculations which are of a 
design nature, and those that are undertaken during the control 
of the physical process. Real time work places a heavy penalty 
on inefficient computational techniques, and boundary value 
methods tend to be among the most inefficient. 


Minimum Squared Deviation and Control Energy 
The performance index is taken to be 


f(x, t) — min 


+ x 2 4 ) dA 

V 


from which the optimal control is unchanged at u 
However, the trajectory equations are now 




Pi ~ ~~ x i 
P f 2=-X 2 

P 3 =z Pi v sin x 3 —p 2 vcosx 3 


with free boundary value conditions. 

The computational problem is not attractive in this case 
because at the terminal point, pf is very nearly always zero and 
tends to be influenced by rounding errors. The optimal trajecto¬ 
ries are shown at C in Figures 2 and 3 and the optimal time to 
reach the origin was 3*29 sec. 


Refining Techniques 

Free boundary value problems of the type of eqn (8) lend 
themselves to reverse time computation from the guessed target 
position at time T. However, owing to the extreme sensitivity 
of the solutions and to some troublesome numerical details, it 
proves far simpler to work forwards from the initial time t. 
Consider the boundary value problem of eqn (8) and assume 
that it is possible to proceed by guessing the initial value of 
p ( t ) and adjusting this according to the error at the terminus. 
Represent the solution (6) for the terminal value of p by 


p(T)=¥lp(t),x(t);t,T-] (11) 


If the initial value is correctly chosen then, clearly, X 1 J will be 
zero. Furthermore, if T is extended, the change in the correct 
p ( t ) will satisfy the differential equation 


dw t 1 dp(0 dw 

d t er 


( 12 ) 
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However, although boundary values for these equations are 
known the matrix [b Wjb Pj ] is not. In general p (T) will be 
non-zero at time T and the magnitude of its error can be evaluated 
by some arbitrary definite function n [p (t)] of the terminal 
boundary values. Small perturbations in the initial value p ( t) 
will affect the value of n according to the expansion 


n(p + Ap) = n(p)+%~-Ap i +0(A 2 p i ) (13) 

However, since n can be defined to have a minimum when 
the boundary value is satisfied, a correction scheme can be found 
by differentiation with respect to one of the Pi : 


dn I d 2 n 

Wt! 


8 Pi 


( 14 ) 


and the corrected value of Pi will be Pi + A Pi . The drawback 
of this simple approach is that the instability of the trajectory 
equations usually makes any function n extremely large and 
grossly sensitive to perturbations. It is, however, very simple 
and easy to code into a routine which could deal with all cases 
likely to arise in practice. 

One way of overcoming this is to reduce substantially the 
time interval T-t. Clearly if this interval were zero the boundary 
values are known to be zero and consequently for a small 
interval a good estimate is available. Thus the procedure is to 
start with a small value T-t, perturb each of the p co-ordinates 
m turn, applying eqn (14) to reduce p (0 to zero, t' being the 
temporary value of T. After one cycle of the perturbations it is 
essential to rotate the axis of the perturbation coordinates to 
make them lie along the direction of grad n. Having solved the 
first stage over the reduced interval, it can be extended and the 
process repeated. As the next guess for the boundary values 
either the previous value can be used or it can be up-dated with 
a crude solution of eqn (12). The matrix [d Wjb Pi ] is now 
available from the results of the previous perturbations and is 
invertible. 

Proceeding in this way, the optimal trajectory can be pro¬ 
gressively extended until it covers the given time interval, 
owever, it may happen that some physical condition is satisfied 


before the full time, i.e. the target region, is entered, and then the 
computation can be terminated earlier with consequent economy. 
This procedure has the advantage that during real time com¬ 
putation a ‘part-time’ optimal solution is always available for 
the current position and this can be used as an approximation 
while its up-dated value is refined. It has the disadvantage that 
errors in the numerical integration tend to disguise the minimum 
sought for, and these increase with T-t. 


A Boundary Value Example 

The method can be illustrated by an example whose non- 
linearities cause difficulties with the more conventional tech¬ 
niques. Consider a dynamic system of the pendulum type 


x"-(x') 2 + x=ni (15) 

Select the control law u to minimize the performance index 


f(x,t, T) = ~ 


fr 


(.x\ + x 2 2 + u 2 )dt 


( 16 ) 


The dynamic system is controllable only in the absence of limits 
on u. Transforming to the usual phase coordinates and forming 
the optimal equations yields the set 


X2 X2 Xj P2 

, ( 17 ) 

Pi = -*i+P 2 

p't.— ~x 2 — Pi —2 x 2 p 2 

If the end point is considered to be free, then the boundary 
values are p (T) zero. Direct computation of the trajectories 
backwards from the terminal region surrounding the origin 
reveals an optimum phase portrait of Figure 5. It shows that 
it is difficult to reach the chosen initial point (x x = 1, x 2 = 0) 
because of the sensitivity of the trajectories. 

At the origin the non-linear system behaves as the linear one: 

x' = A-x 
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Figure 8. Variation of optimal boundary values 


The solution of this system as t tends to infinity, is dominated 
by that of the largest positive eigenvalue of A, and lies parallel 
to the corresponding eigenvector. Thus all solutions tend to the 
one eigenvector through the origin and this implies extreme 
sensitivity when working in reverse time scale from the origin. 

Using the technique described, which has been programmed 
into a series of short routines for the Ferranti Mercury computer, 
a trajectory can be found which satisfies any initial conditions 
and boundary values. Figure 6 gives the final trajectory and also 
the variation of the end point x (T) with the time interval T-t 
given in Figure 7. Figure 8 shows how the computed finite time 
boundary values tend to the Hamiltonian surface H = 0 satisfied 
for the infinite interval. 

Function Space Approximations 

Function space methods are an alternative to trajectory 
methods in deterministic systems, but are the only approach in 
systems containing random components. In some systems the 
algebraic form of f(x, t) is known in advance and the com¬ 
putational formulae may then be put into discrete or continuous 


time form, whichever is most convenient. However, when the 
form of f(x, t) is unknown, only discrete time computing formu¬ 
lae are feasible. This is usually the situation in adaptive systems. 

A Function Space Approximation for an Adaptive System 

To illustrate the use of function space approximations 
consider the following example. A simple regulator contains 
a fixed unknown gain a, in the control path (Figure 9). The 
system is disturbed by a noise which is an independent 
Gaussian sequence with zero mean and variance o. In order to 
set up the problem to lead to discrete time computing formulae 
it is assumed that the control is changed only at unit time 
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intervals, when x is also observed. The dynamic equation for 
the regulator is 

*» = *»-*+aw B _ 1 + f B _ 1 (18) 

The unknown gain a is re-estimated after each observation of x, 
the estimation being made according to the Bayesian formula’ 

posterior density = likelihood x prior density (19) 

Since [f n ] is an independent Gaussian sequence, the successive 
posterior densities can be made Gaussian. The likelihood is 
given by 


l(x n \a) = Qxp 


1 (x n ~x n _ 1 - 




If the prior density is Gaussian with mean m n ~, and variance 
v n~!•> then eqn (19) gives 


W, 




(x,r 




m„ = - 


v„ — - 


-i + 




"n- 1 + 


a 


( 20 ) 


Eqns (20) can now be used to up-date the mean and variance 
after each observation. It is to be noted that they are non-linear 
In order to compute the control at instant n-1, it is necessary to 
have a priori distribution of the mean at the next time instant n 

f ° Und by substituti *g ^r x B as yet unknown, in 
eqn (20) from (18) yielding a stochastic equation 


= - 


+aU «-l + U n-1 C„-l 

-_ n — 1 


2 , cr 


( 21 ) 


The performance index with r stages to go (note r indexes 
time backwards) is taken as 


where 


A=2+ — + 

u 2 2 + ~ 

Vj 


m 2~+cnr 2 +u 2 Q 2 

v 2 


2 cr 
U 2 2 +— 
V 2 . 


B = 1+- 




2 O' 

U q- 

*>2 


m 2 „ ~^ rGC U 2 + U 2 % 2 
V 2 


2 <7 

u 2 2 +— 
V 2 . 


It is evident that no simple analytic expression can be found 
for /j. The complexity of the performance index expression is 
seen to be a consequence of the highly non-linear nature of the 
estimation equations; this is a common occurrence in adaptive 
systems. Now it is simple to evaluate / 2 at any chosen point 
(x, m, v) using a digital computer, but this produces / 2 as a set 
of numerical values; to avoid storing all these points f 2 can be 
condensed into a set of three dimensional orthogonal poly¬ 
nomials. This was done on a small digital computer having 
1,024 word working store using a programme developed by 
Cadwell and Williams 5 . 

Cadwell and Williams’ programme is designed for a particular 
small computer. It uses a modification of Forsyth’s method for 
generating orthogonal polynomials of successively higher order 
using only the previous two polynomials. Owing to machine 
limitations only 200 data points can be fitted in three variables. 
However, in this example, it was found that the mean square 
error could be made less than 0-5 per cent when tested over a 
larger number of points. The polynomials were computed up to 
order 4, involving 35 coefficients of powers of x, m , and v. 

Having approximated f % it is possible to compute / 3 . Since 
the error of approximation is small the iteration for /« may 
be written 


f r (x„ m„ v r )=min E jj) (u 2 + x 2 _ j J (22) 

Note the n in formulae (18) and (20) will also index backwards 
when used in conjunction with (22). The authors regret this 
notational inconsistency. 

A discrete time iteration for the performance index may now 
be set up by using Bellman’s Principle of Optimality 

f r (x n m„ v r )=min E {x z _ k + u z +f r _ u r _ J} 

U r 


— min jh (xq + ( 23 ) 

It is understood that the value of to be used in this iteration 
is the one resulting from the application of u r . 

On substituting for x 0 , and averaging over both £ and the 
current density of oc followed by minimization with respect 
to a,; the analytic expression for u x and f x may readily be found as 


_ — m]*! 

ll mf+uT+1 


fi (xi, m u v x ) o- (24) 
The expression for / 2 , indexing n backwards, is now 


f 2 (* 2 , m 2 , v 2 ) = min E j u\ + (x 2 + <m\ + ^ 2 ) 2 ~ L <j (25) 


f 3 (* 3 > V 2 ) 

= mm E {ul+x 2 2 +f 2 (x 2 , m 2 , v 2 ) + d 2 (x 2 , m 2 , v 2 )} (26) 

where d 2 (x 2 , m 2 , v 2 ) is the error in /„ and f a * are the computed 
values of/ 3 . Now the minimum in f 3 * will be close to that in /, 
so that 3 

ft ~fz + min E {d 2 } (27) 

Roughly, then, the error is additive at each stage. 

To give an idea of the range of the coefficients of the powers 
of x, 7w, v, in/ 3 the largest value was 11-2, then there were nine 
in the range 1-10, eight in the range 0T-1, eleven in the range 
0*01-0*1 and only seven below this. Numerical experiments 
showed that omission of some of the smaller coefficients had 
a serious effect in certain regions of the variables. 

The optimal value of control was found by a gradient 
technique, the expression for /being evaluated for a sequence 
of u values. A defect of the method is that polynomial approxima¬ 
tions tend to show ripples, especially near the end of the fitted 
range, these ripples acted as false minima during the gradient 
computations, and it was necessary to check each minimum by 
approaching it from two sides. However, even this check was 
a little uncertain when the minimum was very flat, in practical 
terms this could make great differences to the control and was 
important. These effects were greatly reduced by approximating 
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the results so that the control was smooth function of x, m, v. 
Once / 3 had been found at a suitable set of points a further 
polynomial approximation could be found and the whole 
process repeated to find / 4 . The polynomial approximation 
method required the same computational process at each stage, 
which is convenient. 

In this example after three stages the control became a 
stationary function of state, some results are shown in Figure 10. 
By trial and error a simple approximation for the stationary 
control was found to be 


fr CO = min E {x r 2 _ x + u 2 r +/ v _ x (x r _ x )} (29) 

V r 

/iCq) = min E{xl + ul} 

V i 


This iteration may be carried out analytically very simply. 
The stationary solution is 


— cmv 
1 + v + cv 


(30) 


- mx 


m 2 +—v 2 +1 


for x, m and v in the range 0-5-0. 


(28) 



X 


where c is the positive solution of 


c 




+ 


1 + v 


m 


( 31 ) 


On comparing the resulting values of control from eqn (28) 
and (30) it will be found the stochastic control is 10-20 per cent 
smaller than the adaptive control. This is to be expected on 
physical grounds, since the adaptive control has an exploratory 
element. However, the stochastic control value is a reasonable 
approximation to the adaptive one, considering the immense 
difference in the amount of computing involved. 

At the present time the best approximation method for 
adaptive control known to the authors is the replacement of the 
fully adaptive system by the partial one as above. Since each 
unknown parameter often involves two, or more statistics, 
depending on the distribution used, the reduction in dimen¬ 
sionality can be substantial. It is usually apparent on physical 
grounds that the approximation will be a valid one. In many 
cases the accuracy of approximation will be better than in the 
example above. 


A Function Space Approximation for Deterministic Systems 

In non-linear deterministic systems a function space approx¬ 
imation can often be a useful alternative to the trajectory 
method. These methods depend on having an analytic form for 
the performance index of an approximated system. First it must 
be possible to rewrite the dynamic equation of the system in 
linearized form as 

x — A (x, f) x+B(x, t) u (32) 


Figure 10. Stationary control law for adaptive regulator To obtain an analytic form for the performance index choose 


On the particular computer used (300 [Lsec multiplication 
and 4 msec division time) each point involved about 1 min of 
computation and over four stages some 800 points were required, 
needing in all some 14 h of computation, it is therefore inter¬ 
esting to see what further method of approximation could be 
used so as to reduce this computation load. 

An alternative possibility is to replace the system by one which, 
on physical grounds, would appear to have a similar control 
solution. The simplest alternative system is to regard a as a 
random variable with a fixed distribution at each stage, neglect¬ 
ing for the moment the transitions in mean and variance. Thus 
at each stage the only variable to be considered is x. The mean m, 
and the variance v, of the estimate of oc 9 are then successively 
up-dated and used in the solution for u (30). 

The functional iteration for the stochastic system is a func¬ 
tion of x only 


fix, 0 = min 

w 


CO | 

- (x t Qx + u t Ru) • dr 
o 2 


(33) 


where Q and R may be general time dependent matrices. 
However, for simplicity, take them to be constants in this 
description. The partial differential equation for /(jc, t) is 

— ^ = min[x T ~x + u T ~w +p T (A(x, t)x + B(x, t) u)~\ (34) 
Ct u A z 

On minimizing and regarding A , and B as constant it is found 
that 

M=-ii _1 B(x,f) T P (35) 

where 

P=lPuP2--PnY with p t = 
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After substituting for the optimal control it will be found 
that eqn (34) can be solved by substituting f(x, t) =\x T P x 
where P is the solution of a matrix Ricatti equation: 

P' + A t (x, t) P + PA(x, t) + Q = PB (x, t)R~ l B T (x, t)P 

Utilizing the stability properties of the Ricatti equation 7 it can 
be shown that P is the positive definite solution of 

PA (x, t ) + A T (x, t) P + Q = PB(x, t)R~ 1 B (x, t) T P (36) 

The approximation scheme now uses eqn (36) to solve a 
state dependent matrix. The resulting matrix is substituted into 
eqn (35) to compute the control vector. The advantages of this 
method are (a) the solution requires only algebraic computation, 
and uses currently available quantities j (5) the precise nature 
of A and B is unimportant, thus the method is readily extendable 
to an adaptive scheme, where A and B vary as the result of 
measurement, and (c) it can be shown that an appropriate 
choice of the linearization will always result in a stable con¬ 
trolled system. 

The stability and ease of realization of the resultant con¬ 
trolled system are probably the most important practical 
factors in favour of this technique. To show the stability, con¬ 
sider the general second order system 


x\=a 12 (x 2 ) x 2 


X 2 ^21 ( x l) X 1 A~ d 22 ( x i, X 2 ) X 2 -j-du (37) 

the factors a 2l (x^ and a 22 (*i> * 2 ) can be any bounded functions, 
but a 12 (* 2 ) must satisfy certain requirements given below. 
Take the performance index 


j\x, t) = min 


( x i A-xl + u 2 ) dr 


(38) 


Applying eqn (36) the elements p u of P satisfy 

d P 11 P 12 ~~ a i 2 Pn ~( a 2 i A-a 22 )p 12 =0 

d2 Pi2~2a 21 p 12 -l=0 (39) 

d 2 pl 2 -2a 22 p 22 -2u 12 p 12 -1 = 0 

On finding the stable solution of eqns (39) it is possible to 
evaluate the control 


u=-d{p l2 x 1 +p 22 x 2 ) 

where p 12 and p 22 are the positive solutions of eqns (39). 

On substituting for u in the original equations the controlled 
system has the form: 

x l~ a 12 (^ 2 ) X 2 

x 2 = ~b 2 i(x i )x 1 — b 22 (x 1 ,x 2 )x 2 (40) 

where the functions b 21 and b 22 are the positive roots of 

^21 ( x i) — \_ a ii C*i) A- d 2 ~\ z 

b 22 x i) = [_ a 22 ( x i> x z) A-d 2 (2a 12 (x 2 ) Pi 2 + 1)]’ 2 ~ (41) 

The stability of eqns (40) can now be established by applica¬ 
tion of the second method of Liapunov 6 . Consider the proposed 
Liapunov function 

r* 1 rx 2 

K(x)= wb 21 (w)dw+ wa 12 (w)dw (42) 
Jo Jo 

Clearly V (x) is a positive function which is bounded if a 12 (x 2 ) 
is positive and suitably restricted. Its time derivative is 

V = — b 2 2 (^ 1 ^ 2 ) ^12 fe) x 2 (43) 

which is negative semi-definite. If the x 2 axis is not a permissible 
trajectory of the system of eqn (40) then the expression of 
eqn (42) is a Liapunov function of the system. This function 
defines a series of closed surfaces over the whole phase space 
about the origin which the trajectories enter. Thus the system 
is asymptotically stable about the origin. 

A particular case which has been studied computationally is 
the Van der Pol equation 

x "A-a(l~x 2 )x' + bx = du (44) 

This equation was linearized in phase space form by putting 

x'i ss x 2 9 012 = 1> a 21 — b, a 22 =-a( l—x\) (45) 

Using the performance index of eqn (38) some comparisons of 
the computed trajectories in the exact and approximated cases 
are shown in Figure 11, which gives the phase space trajectories 
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and the necessary control signals for the exact and approximated 
cases, Figure 12. The closeness of the approximation occurs in 
many practical cases, and is an indication of the effectiveness 
of the method. To illustrate the implementation of the scheme 
an analogue computer arrangement is shown in Figure 13, which 
solves eqn (39). 




Figure 13. Analogue solution of sub-optimal system 


The degree of approximation can be improved by varying 
the performance index slightly (i.e. Q and R). Figure 4 shows 
the effects of such variations and Figure 11 suggests that a 
system linearized and optimized with respect to 

/(*,0-[ (j x l+ y*a+Y“ 2 ) dT 

closely approximates the non-linear system optimized with 
respect to 

1 P 00 

f(x, 0=yj (Xi+X 2 +W 2 )dt 

Thus it appears possible to rescale the approximate phase plane 
to fit the optimal one by adjusting the performance index 
appropriately. 


Conclusions 

The general mathematical equations for optimal and adaptive 
control can now be set up, but comprehensive methods for 
solution are not known. Approximation methods are being 
developed, but they must be used according to the individual 
circumstances of each problem. A number of different possible 
techniques of approximation have been given with appropriate 
examples. 

The first point stressed is that the mathematical setting up 
of the problem can often be varied so as make the computation 
easier, whilst still giving a satisfactory physical solution. The 
second point is that the methods can be grouped into two 
classes, trajectory methods applicable only to deterministic 
systems, and the function space method which is of wide 
application. A systematic search technique for solving trajectory 
problems has been described. A function space method using 
orthogonal expansions and another using linearized equations 
has also been given. 

Computing was done at the University of London Computer 
Unit. 

Nomenclature 

oc Unknown gain factor 
A Term in dynamic equation 

B Term in dynamic equation 

d Control coefficient 
f(x,t) Performance index 
H Hamiltonian type function 

/ (x a) Likelihood function 
L Integrand in performance index 
m Estimated mean value 
n ( p ) Norm function 
p Co-state vector 

P Matrix in expansion of performance index 

Q Cost of state matrix 

R Cost of control matrix 

t, T, Time 

u Control vector 

v Estimated variance 

V (or) Liapunov function 

x State vector 

a Variance 

{f n } Noise process 

v Lagrange multiplier 
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DISCUSSION 


A. R. M. Noton, Electrical Engineering Department, University of 
Nottingham, Nottingham 

The authors have described the difficulties associated with the solution 
of the two-point boundary value problem, following from the appli¬ 
cation of Pontryagin’s principle to deterministic systems. The trajec¬ 
tories are grossly sensitive to perturbations in the initial conditions 
and neither the digital nor the analogue computer provides a convenient 
method of solution. 

In such computations the solutions are unrolled as functions of 
time but, to solve the two-point boundary value problem, another 
approach has been suggested 1 . The set of differential equations in x 
and p are approximated by their finite-difference form, i.e. the dif¬ 
ferential equations are replaced by a finite number of algebraic equa¬ 
tions. These equations can then be solved on an analogue computer 
employing a corresponding number of operational amplifiers. In many 
practical cases, however, a significant saving on the required number 
of amplifiers can be achieved by preliminary algebraic manipulations 
and eliminations. The boundary values can easily be inserted and the 
method seems particularly attractive for approximating the required 
computations, at least on an on-line basis. Have the authors considered 
this approach ? 

Reference 

1 Kipiniak, W. Dynamic Optimization and Control . 1961. New York; 
Wiley 

J. H. Westcott, J. J. Florentine and J. D. Pearson, in reply 

The amount of analogue equipment necessary for the matrix technique 
appears to be excessive in general. However, this method may have 
special applications. 

F. Kylstra, KoninklijkejShell Laboratorium, Badhuisweg 3, Amster¬ 
dam, Nederlands 

For the particular system described by eqns (9) the authors demon¬ 
strate the insensitivity of the solution to the formulation of the per¬ 
formance index. The fact is that this obviously holds in case transit time 
is the criterion. Is there some evidence that may lead one to generalize 
this observation to the point where, for a whole class of problems, one 
would be justified in employing a convenient index rather than the 
exact one, without trying both? It seems that this particular system 
already offers a counter example in the case of the control energy 
criterion. 

J. H. Westcott, J. J. Florentin and J. D. Pearson, in reply 

Our point is that this is the case for some problems and is worth an 
investigation. Our criterion in this example was the ‘similarity of 
trajectories’ for various performance indices. 


second-order non-linear systems. The approximate solution was found 
to be very sensitive to the digital increment. No satisfactory result 
could be obtained. 

Reference 

1 Kalaba, R. On non-linear differential equations, the maximum 
operation and monotone convergence. J. Math. Mech. 8 (1959) 519 

J. H. Westcott, J. J. Florentin and J. D. Pearson, in reply 

We have tried quasi-linearization techniques since this paper was 
written, and obtained some success. Digital computer solutions of 
non-linear differential equations are always ‘sensitive to the digital 
increment’. The examples quoted gave us no difficulty. 

H. Nour-Eldin, Swiss Federal Institute of Technology, Gloriastrasse, 
Zurich 6, Switzerland 

I congratulate the authors on their interesting paper. It is really worth 
while to show the possibilities and limitations of the theory in solving 
concrete problems. In using the function space technique (dynamic 
programming), the authors overcame the shortage of the computer’s 
storage by approximating the decision function by a three-dimensional 
orthogonal polynomial. They also showed that neglecting the transits 
in mean and variance will reduce to one variable which is the phase 
coordinate X. I would like to remark that the authors have a first-order 
system as an example. If the system is of higher order, one has to use 
Lagrange multipliers which are functions of time. The number of these 
multipliers will increase as the phase coordinates of the system increase 
and I feel that this is the real handicap of setting the problem for 
digital computers. Can the authors say whether any success regarding 
this point was achieved ? 

Furthermore, regarding optimal systems having performance in¬ 
dexes of the form of integral of sign definite functions (V), I think 
some success can be achieved by constructing the Liapunov function 
corresponding to this sign definite function (V). The solution of linear 
system leads to a system of algebraic equations. The authors have 
shown the advantages of this method. I mention that this method can 
be applicable to systems non-linear in phase coordinates but linear in 
the control variable. The differential equations can be written in the 
form of 

X=F(X)+BU 

One can find the Liapunov function for the unperturbed motion satis¬ 
fying the equation 

n r)W 

( E 

with V sign definite function and the performance index 


G. C. Agarwal, C.LS.L. School of Electrical Engineering, Purdue 

University, Lafayette , Indiana, U.S.A. 

The authors have not given any reference to the quasi-linearization 
technique 1 for numerical solution of two-point boundary value prob¬ 
lems. This technique is computationally very efficient with quadratic 
convergence property and can be applied to non-linear and also partial 
differential equations with multipoint boundary conditions. The 
method is particularly suitable for the numerical solution of the set of 
differential equations of the type in eqn (6). Many examples have been 
worked, using this method, at Control and Information Systems 
Laboratory, Purdue University, U.S.A. 

The linearization technique eqns (32) and (36) which was proposed 
earlier (Reference 7 of the paper), was tried by me for a few first- and 


1 = 


*t 

o 


Vdt 


This equation can be solved, using Zubov’s method of construction 
of Liapunov functions, as long as the system differential equations 
fulfil Zubov’s requirements. 


J. H. Westcott, J. J. Florentin and J. D. Pearson, in reply 

We agree that the selection of Lagrange multipliers is a problem. The 
difficulty of a more exact analysis using Zubov’s method, for example, 
is that the complexity of the controller increases rapidly with the degree 
of Zubov’s polynomial expansion. 
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V. W. Eveleigh, General Electric Co., 3-118 Electronics Park, Syra¬ 
cuse, N.Y., U.S.A. 

The authors have presented an interesting summary of techniques 
available for solving optimization and adaptive control problems. The 
examples illustrating how a solution may often be obtained by proper 
choice of the performance criterion are particularly well chosen. 

The memory requirements often encountered in the application of 
dynamic programming techniques, and the possibility of using a poly¬ 
nomial approximation to overcome this difficulty, as indicated by the 
authors, has also been pointed out by Aoki 1 , Kipiniak 2 , Peterson 3 , and 
Merriam 4 , among others. Unfortunately, as also pointed out by the 
authors, very little has been accomplished in developing a theory of 
error for these approximate techniques. Work in this direction may 
well prove fruitful in the future. 

In the area of computational techniques, I would like to indicate 
an entirely different line of development than that referred to in the 
paper. It is currently possible, using any one of several alternate 
techniques available, to obtain the optimum solution to a very general 
class of non-linear and/or time-varying differential equations with 
boundary values at two or more points. The method involved an 
iterative procedure based upon linearization of the equations about 
a nominal (non-optimum) solution. A form of this method was perhaps 
first conceived by JBreakwell 5 , but Kipiniak 2 also suggests a similar 
technique. Brcakwell’s original suggestion was to proceed to the 
optimum solution in one iteration, but it is often found that this 
approach fails due to the range of extrapolation required. Kelley 6 and, 
independently, Bryson and Denham 7 , have developed iterative tech¬ 
niques based upon linearization of the solution about the nominal path, 
but in which the size of the improvement is constrained to assure 
efficient convergence to the desired solution. More recently, Merriam 4 , 
and Breakwell, Speyer and Bryson 8 , have developed second variation 
techniques with improved convergence capabilities near the desired 
optimum. Also, techniques for dealing with inequality constraints upon 
one or more elements of the state vector have been developed by 
Dreyfus 0 , and by Bryson and Denham 10 . An excellent set of examples 
illustrating the use of these techniques is available in the literature. 
Each of the methods, with minor additional development or inter¬ 
pretation, gives rise to a control law designed to force the system to 
follow an optimum or near optimum trajectory through phase space. 
Analytical control examples are also available, but few, if any, actual 
hardware applications have been reported. 

I hasten to point out that this short bibliography of computational 
techniques is not offered in criticism of the author’s survey of tech¬ 


niques available, but only as a supplement. I am fully aware of the long 
lead time required between the work upon which a paper of this type 
is based and its presentation at the conference. The authors are no 
doubt already aware of some or all of these references and have 
found, or will find them most interesting. 
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J. H. Westcott, J. J. Florentin and J. D. Pearson, in reply 

Dr. Eveleigh questions the stability of the approximation procedure 
of replacing an adaptive system by a stochastic one. In use, the un¬ 
certainty, i.e. the variance of the estimated parameters, would be 
artificially increased. This would lead to a control less dependent on 
the numerical values of the parameter estimate, thus counteracting 
instability. 

We thank him for his additional survey of current computational 
methods. 





An Optimal Guidance Approximation for Quasi-circular 

Orbital Rendezvous 


H. J- KELLEY 


Summary 

Three-dimensional guidance about a time-optimal rendezvous flight 
path is examined within the framework of a quasi-circular orbital 
dynamics assumption. The guidance scheme, optimal in the same 
sense as the nominal trajectory, is based formally upon the approxi¬ 
mate construction of a field of neighbouring optimal rendezvous 
paths, following a method developed by Kelley in an earlier paper 1 . 

For purposes of illustration, a class of reference trajectories, 
generated by a low magnitude, circumferentially directed thrust vector, 
has been adopted for the guidance analysis. Such trajectories are time 
optimal, under appropriate circumstances, for transfer between 
neighbouring co-planar circular orbits. The application considered 
permits an analytical representation of the extremal field which in 
turn leads to a closed form linear feedback control solution with 
time-varying gains. 

Some suggestions are also given for possible modifications which 
might enhance system accuracy and the range of operability during 
practical implementation of low-thrust rendezvous guidance. 

Sommaire 

On examine le probleme du guidage dans les trois dimensions au voisi- 
nage d’une trajectoire de rendez-vous optimale par rapport au 
temps, dans le cadre de la dynamique des orbites quasi-circulaires. 
Le schema de guidage, qui est optimal dans le meme sens que la 
trajectoire nominale, est formellement base sur la construction ap¬ 
proximative d’un champ de trajectoires de rendez-vous optimales 
voisines, suivant la methode exposee par M. Kelley dans un article 
recent. 

Comme exemple, on a adopte pour l’analyse du guidage une 
classe de trajectoires de reference produite par un vecteur circonferen- 
tial defaible poussee. De telles trajectoires sont, dans des circonstances 
appropriees, optimales par rapport au temps pour le transfert entre 
des orbites circulaires voisines dans le meme plan. L’application 
consideree autorise une representation analytique du champ ex¬ 
tremal qui, a son tour, mene a une solution de commande sous forme 
hneaire fermee a reaction, avec gains variant dans le temps. 

En conclusion, on presente quelques suggestions, qui ont, pour 
objet d’ameliorer la precision du systeme et le domaine de fonction- 
nement dans la realisation pratique du guidage de rendez-vous a 
faible poussee. 


Zusammenfassung 

Das Problem der 3-dimensionalen Lenkung, um eine zeitoptimak 
Flugbahn fur Begegnungen von Raumfahrzeugen (Rendezvous-Tech- 
mk) zu erzielen, wird im Rahmen einer quasi-kreisformig angenom- 
menen (dynamischen) Bahn untersucht. Der Lenkvorgang, im gleichen 
Smne optimal wie die Bezugsbahn, stiitzt sich formal auf die ange- 
naherte Konstruktion eines Bereiches (Bundel) von nebeneinander- 
liegenden optimalen Begegnungsbahnen; diese Methode wurde in 
emem friiheren Aufsatz von Kelley entwickelt. 

ZurErlauterung warden eine Reihe von Begegnungsbahnen, die 
em m Richtung der Peripherie zeigender Schubvektor von geringem 

fUr - d !f Untersuchun g der Lenkung angenommen. 
Solche Flugbahnen smd zeitoptimal, sie eignen sich unter entsprechen- 

H P LnH SUn T n f “ Uberwechseln zwischen zwei nebeneinander- 
uegenden, m der gleichen Ebene beflndlichen kreisformigen Bahnen. 


and J.C. DUNN 


Die betrachtete Anwendung lafit eine analytische Darstellung des 
extremen Bereiches zu, was auf einen Regelkreis mit zeitverander- 
licher Verstarkung fiihrt. 

Der Aufsatz enthalt einige Vorschlage fur mdgliche Abanderungen, 
die die Genauigkeit und den Operationsbereich wahrend der prak- 
tischen Lenkung mit geringem Schub erhohen konnen. 


Introduction 

The second-order guidance approximation scheme employed in 
this paper has been developed in an earlier publication 1 . Essen¬ 
tially, the idea is to select a flight path, optimized in some 
appropriate sense, as a nominal trajectory, and then to base 
guidance upon a family, or ‘field 5 , of optimal trajectories 
approximated in the vicinity of the nominal. 

The investigation, as is shown Jater, applies this scheme to 
the guidance problem for orbital rendezvous. The method is 
tractable for a fairly wide class of problems, although in general 
it must be carried through numerically. However, in the present 
case an analytical treatment becomes feasible because of the 
recent availability of a particularly simple optimal transfer 
manoeuvre suitable for use as a nominal trajectory. The nominal 
manoeuvre, as it appears in this paper, is a direct outgrowth of 
a co-planar circular orbit transfer analysis conducted by Hinz 2 . 
The problem posed by Hinz, although phrased somewhat dif¬ 
ferently with regard to coordinate systems and assumptions 
employed in deriving the equations of motion, is mathematically 
equivalent to the nominal transfer manoeuvre problem inves¬ 
tigated herein. Both cases yield to an analytical treatment of the 
boundary value problem whenever the manoeuvre duration is 
an integral multiple of a reference orbit’s period. 

The analysis of three-dimensional rendezvous guidance for 
the class of trajectories discussed above leads directly to a 
synthesis of a linear feedback control solution with time- 
varying gains given in closed form. 

Some suggestions are also given for possible modifications 
which might enhance system accuracy and the range of opera¬ 
bility during practical implementation of low-thrust rendezvous 
guidance. 

The Differential Equations of Powered Flight 

Considerations begin with the differential equations of three- 
dimensional powered flight in a central inverse-square force 
field (. Figure 1) : 

k F 

f-r cos 2 iJ/9 2 - rxjj 2 +-j=— vcos 3 sin a 
r m ' 

** rd . F 

0+2 ~ —2tani/f -r-vcosficosa 

f mr cos \j/ (X) 
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^ + 2 —+ 4”Sin2i/A0 2 = —-vsin 8 
r 2 mr 

vF K n 71 

m= _._ ; — 0<v<l 


Finally, if note is made of the fact that Jc/RqCoI = 1 , then 
eqns ( 1 ) can be put into the following reduced first-order form: 


= (1 + 17 ) (1 + n ) 2 cos 2 ip — 


1 


+ 7 


(i Frj) 2 (1 + 0 


- v cos ft sin a 


where F is the maximum thrust level of the reaction engine; C 
is the propellant exhaust velocity; v is a throttle variable; and 
m is the instantaneous vehicle mass. The difficulty in obtaining 
any sort of particular solution for these equations needs no 
comment here, except that it provides a motive for the simpli¬ 
fications which are now introduced. The object is to devise 
certain assumptions which will allow the replacement of eqns ( 1 ) 
by an approximate set of differential equations which are linear 
in the state variables r, 0 , and \j/ 9 and their time derivatives and, 
preferably, separable in the control variables v , <x 9 and p. 
(Simplifications of this kind are required to make flight path 
optimization and guidance problems analytically tractable.) 
To be specific, it is preferable that these approximate differential 
equations describe low-thrust acceleration transfer trajectories 
between neighbouring circular orbits. 



The following set of dependent and independent variable 
transformations will prove useful for our purposes. Let 

H/)=tf o P+f?(0] 


2 u (1 + v) 
1 +!? 


+ 2 w (1 + u) tarn/' 


p* 

+ (l+ f7 )(l + ^ 


v cos/? cos a 


, 2liW 1 , 2 • ~ / F* 

"' = -T+^T (1+, ’ , sin2, ' ,+ (i+,)(i+{) 


v sin P 


rj —u. 


( 2 ) 


8 =V 


Ip' = W 



where F * = F/m 0 R 0 wl is the reduced maximum thrust accelera¬ 
tion; C* = C/R 0 w q is the reduced exhaust velocity; and 
the superscribed prime denotes differentiation with respect to 
the reduced time, r. 

Now let us assume that F* 9 £, u, v , w 9 1 ], and \p are all terms 
of order d or smaller (d < < 1). Under these circumstances, one 
would therefore anticipate that all terms of order 3 2 in eqns ( 2 ) 
will become negligible with respect to terms of order 3. Thus 
the following simplified differential equations are arrived at: 


u' = 2v + 3rj + F*v cos p sin a 

*/= — 2 uH-F*vcos p cos a 

w/= — i/f + F*vsin P (3) 

Y \ = U 


0(/) = r(0 + e(0 


= v 


and m(t) = m 0 [ 1 + £(*)] 

where R Q is the radius of a circular reference orbit situated in 
the \p = 0 plane (Figure /); m 0 is some reference mass; and r 
is a fictitious angle defined by the differential expression 


1=0)0, T(0)=0 

where o) {) is the reference orbit’s period. 

Furthermore, since r(t), as defined above, is a monotonic 
time-like parameter, it is permissible to change the independent 
variable in eqns (1) from t to r. This can be accomplished by 
simply relating t derivatives to t derivatives as follows: 

d__dr d_ = d_ 
d t d t dr 0)0 dr 


d 2 

d? 


_d 
d t 



= co 0 


dr 2 


1 y=w 


It may be said that, to the first order of small quantities, eqns 
( 3 )* are descriptive of quasi-circular flight for the following rea¬ 
son; if, as has been assumed, u, v , rj 9 etc. are of order <5, then it 
follows at once that 




= 0(5) 


where E and h are specific energy and angular momentum 
respectively. Consequently, the energy-momentum images of 


* These equations are identical in form to the differential equa¬ 
tions of Wheelon 3 and Anthony 4 . However, the dependent variables 
and thrust vector steering angles are not subject to the same inter¬ 
pretation. In particular, the quantity s in eqns (3) is not required to 
be small—an important point in the subsequent development. 
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trajectories which are adequately described by eqns (3) should 
everywhere be close to the locus of circular orbits in the E-h 
phase plane (Figure 2). 

Clearly, the validity of the quasi-circular differential equa¬ 
tions will be compromised when the parameters 1F* 9 F*/C* 9 and 
r exceed certain critical values. Just precisely what these critical 
values are cannot be determined until the nature of the control 
schedule v(r) 9 oc( t), and /?(r), is specified. The reader is advised 
to bear this in mind in the sequel. 



Figure 2. Energy-momentum phase diagram 


Optimal Transfer Between Neighbouring Circular Orbits 

The optimal orbit transfer problem may be stated as follows: 
given two neighbouring circular orbits, find the steering angles 
oc(z) and /3(r) and the throttle schedule v(r ) which produce a 
transfer between the two orbits in minimum time. For present 
purposes, the case for which the terminal orbits are co-planar 
will be selected, a class of optimal transfer paths within the 
framework of the quasi-circular dynamics assumption derived, 
and later, these paths employed as nominal trajectories for the 
three-dimensional rendezvous guidance analysis. 

To reiterate, if the subscripts 0 and / denote initial and final 
conditions respectively, a search is made for a set of control 
functions oc( t), /?(t), and v(r ) which minimize r f9 produce a 
state transition which evolves in accord with eqns (3), and 
which satisfies the circular orbit boundary conditions, viz: at 
r = 0, u = v = w = 7] = e= \l/ = 0, £ = 0; at r = r f 

u — w = 2v + 3rj = \[/ = 0, ')] = K (const.). The problem so 
stated is a Mayer variational problem with bounded control 
variables. The necessary conditions to be satisfied by its solution 
are well known and are written here for the present application 
without further comment: 

Let 

Xi = state variables, u, v , w, rp e , x[/ 9 £ 

y k = control variables v, oc , /3 

h = multiplier functions 

Ai = undetermined constant multipliers 

P = function to be extremized = r f + A x (u f ) + A 2 


8 H 

(6) 

, 6H r 

x i=0l [ ec l ns ( 3 )] 

(7) 

together with the corresponding natural boundary and trans- 
versality conditions, 

1 

<o Ico 

1 

II 

(5a) 

A - SP 
" 0* t/ 

(6a) 

x io =0, i = 1,...,7 



x lf = x 3f —2x 2f +3 x 4f = x 6f = 0, x 4/ = K (7a) 

Now, a minimum of H is attained at a minimum of H l9 
where H x is that part of H which depends on the control vari¬ 
ables, y. For the problem here, 

H x =F*v cos /? sin a + 1 2 cos cos a + A 3 sin f} - (8) 

Flowever, note that the final mass, m 0 (1 +£ f ) does not appear 
in the pay-off, i.e., the final mass is left open. However, only 
those trajectories for which ||/| is of order <5 are admissible 
because of assumptions implicit in eqns (3). Therefore, eqns (6) 
and (6a) imply that A 7 = 0. Consequently, eqn (8) simplifies to: 


H != F*v (A x cos /? sin a +A 2 cos /? cos a +A 3 sin /?) (8 a) 

The requirements on the control variables oc, (3, and v are de¬ 
termined by reasoning as follows. 

Since a is unbounded, ilHJboc = 0 and blHj/bx 2 > 0 at the 
minimum of H± and hence, 

sin a = - A 2 /(A \ + Af)* cos a = - A 2 /(Aj + A|)^ (9) 

H x reduces to: 

H X = F v[ (A^ + A^)^ cos/?+A 3 sin/?] (10) 

which can be written in the form 


H x —F*v (Aj -j-A 2 +A 3 ) t sin (/? — (p) 


(p — sin 


Cju +^2)^ 
(Aj + Al+Af)* 


( 11 ) 


By virtue of the fact that sin cp > 0, it follows that the principal 
value of (p lies between 0 and n. This last conclusion, together 
with eqn (11), permits the deduction that the minimum of H 1 
occurs when ft = <p — ctz/2*, i.e., when 


sin/? = 


^3 _ 

(Af+Af-hAf)^ ’ 


COS 


qj+Aj)* 

(Ai+A§ + A?)* 


( 12 ) 


(2v f +3ti f )+A 3 (w f ) + A 4 (ii f -K)+A 5 (\j/) (4) 

H = Hamiltonian function = 2 

i 

Then the following equations and inequalities must be satisfied, 

H(y)>H(y) (5) 

i.e., the optimal control y minimizes the function H. 



and H x reduces still further to: 

H x = — F*v(A^ + -j- A 3 )^ (13) 

* Notice that H x is also stationary with respect to this value of /?, 
i.e., (fiHJDfip** 0. 
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From eqn (13) it follows immediately that v = 1 minimizes H x 
whenever 

(Aj + A 2 "H A^) 2 (14) 

Furthermore, it can be verified [by solving eqns (6)] that 
Af + Af +Af # 0 except at a finite number of points on any 
r interval of length 2 n. Thus the indeterminate values of v 
corresponding to Af + A| + == 0 form a set of measure 

zero and our problem is therefore well behaved. 

In summary, the optimal control variables depend upon the 
multipliers A x , A 2 , and A 3 in the following manner: 


sin a = 


sin/? = 
v = 1 


(Ai + A 2 ) 2 

_ ^3 _ 

(A^ + A 2 + A3) 2 


cos cc — • 


W + AS)* 


cos /? = 


{X\+X\ + Xlf 


(15) 


The A’s in turn depend on the unknown Lagrange multipliers, 
A t through eqn (6), and the solution of the boundary value 
problem turns upon one’s capability to solve for these undeter¬ 
mined constants. However, before attention is directed to the 
boundary value problem, it is worth while to emphasize two 
points. First, it is easily demonstrated that the optimal control 
variables given by eqns (15) satisfy a strong form of eqn (5), i.e., 


where A is a matrix of constant coefficients and g is a vector 
whose elements depend upon the control variables oc, ft, and v. 
Solutions for eqn (17) can therefore usually be phrased in terms 
of a fundamental solution matrix 0 (r f , r) and superposition 
integrals, e. g.: 

x(r / ) = $(T / ,0)x(0) + j* o $(tj,-) g[y(f)]dr (18) 


Eqn (19) 


where 


to * * *5 to 7 


g t = F*v cos p sin a = • 


A if 


(A l+X 2 2 +Xlf 


g 2 = f*vcos/3cosa= — - 


X 2 F* 


(X 1 4-A 2 + A 3 ) 


2\i 


g 3 =F*vsinj3= - 


X-lF 


(X\ + X\+Xlf 


( 20 ) 


?4=g5 = g6 = 0 


y^y ( 16 ) 

But eqn (16) together with the linear character of eqns (3), are 
sufficient conditions for a strong relative minimum of P. Thus 
it is assured that the control law of eqns (15) will provide a 
time-optimal transfer between two co-planar circular orbits 
which is unique whenever the boundary value equations possess 
a unique solution. Second, it should be noted that the result 
expressed by the last of eqns (15) is independent of the multiplier 
functions A h and is therefore insensitive to the boundary con¬ 
ditions. Thus a full throttle operational mode is a characteristic 
of the entire extremal field of quasi-circular transfer trajectories 
for the problem of minimum time transfer with final mass open. 


The Orbital Transfer Boundary Value Problem —a Special Class 
of Solutions for Rendezvous 

The differential equations, eqns (3), when written in a 
symbolic matrix notation, have the following structure: 

x' = Ax + g(y) (17) 


gi = 


C* 


As already pointed out, the A’s depend on the undetermined 
constants A t in accord with eqn (6 a). Furthermore, it is noted 
that in matrix notation, eqns (6) have the form 


A'= —A t X 


( 21 ) 


and thus are adjoint differential expressions for eqn(17) [i.e., 
eqns (3)]. Consequently, their solution is determined when 
0 (T/, r) is known, i.e., 

A(r) = ^ T A(T / ) S <? T |^-| (22) 

where 


8 P 
dx lf 


= A 


1> 


-—2 An etc. 

8x 2/ 


In view of these considerations, the co-planar circle-to-circle 
transfer boundary conditions [see eqn (7)] are: 


* Eqn (19): 


cos£ 

—2sinT 
0 




sin'r 

—2(1-cos?) 
0 
0 


2 sin? 

—(3—4cos?) 
0 

2(1—cos?) 

— (3?—4sin?) 
0 
0 


0 3 sin? 

0 —6(1 —cos?) 

cos? 0 

0 (4 — 3 cos ?) 

0 — 6(?—sin?) 

sin? 0 

0 0 


0 0 0 

0 0 0 

0 —sin? 0 

0 0 0 

1 0 0 

0 cos ? 0 

0 0 1 


( 19 ) 
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where: (a) r 0 is the point in time at which corrective action is 
initiated; ( b ) the SxyS are dispersions which would occur at the 
terminus of the nominal trajectory in the absence of corrective 
guidance; ( c ) 8t f is a variation in the final time, x f (Sr f < < 1 ); 
and (d) the coefficients of Sr f in each equation are to be inter¬ 
preted as state variable closing rates, i.e., differences between 
vehicle and target state variable derivatives on the nominal 
path at r = x f (notice that only v has a non-zero closing rate). 

A total of six equations linear in the six unknowns C l9 .. C 5 , 
and Sr f are at our disposal. Their solution may be obtained 
most conveniently by first combining the second and fourth of 
eqns (34) to obtain the result, 

Sv f +2 Stj f =-F*Sx f ( 35 ) 

which may then be used to eliminate Sr f . The remaining equa¬ 
tions can then be written in matrix notation (see Appendix), e. g.: 


and 



(36) 


(37) 


where A (r 0 ) and B (r 0 ) are matrices whose elements depend 
upon r 0 and / and K are constant matrices. But the Sx t are 
related to the <5x i(t0) through eqn(19). Therefore the constants Q 

may be expressed as functions of the ‘initial’ excursions at time 
t 0 , i.e.: 



J*i( T 0 ) 


Su 

Sv 

Srj 

Ss iro 


det \A\ 7^0 


(36 a) 



=B~\t 0 )K$ 2 (x 0 ) 



det |jB| t^O 


(37a) 


Considerations on Rendezvous Guidance Implementation 

In this section, several ways to enhance system accuracy 
and range of operability in practical implementation of low- 
thrust rendezvous guidance are suggested. 

First of all it is noted that real time generation of the system 
gains and the state variables along the nominal trajectory will 
be required in practice. In the case of command guidance, these 
* would presumably be calculated ‘on the fly’ by a ground-based 
computer, whereas if a vehicle-borne computer were employed, 
these quantities would more likely be stored in polynomial 
approximation. Now, with regard to the computation of gains, 
it is worth calling attention to the advisability, in either case, of 
employing double precision arithmetic in the calculation of the 
determinants of the Appendix, since they tend rapidly toward 
zero as time-to-go approaches zero. This, of course, does not 
necessarily imply the need for a highly precise representation 
of the gains themselves. 

As concerns the generation of the state variables along the 
nominal trajectory, it should be pointed out that over the 
course of many revolutions there will be an accumulation of 
error stemming from higher-order effects if the linear equations 
are used. This error will place an additional burden upon the 
guidance system. The situation may be alleviated somewhat by 
introducing second-order corrections to the nominal in the 
following approximate fashion. Second-order terms of the first 
two equations of the system (5), which are significant along 
the nominal, are 

P — ~3rj 2 +2r]v + v 2 
q~ —luv + luY] — F*rj 

If p (0 and q (t) are estimated by insertion of the nominal 
values of u 9 v, and rj as computed via the linearized equations, 
corrections du C9 dv C9 Sr\ c and de C9 corresponding to the forcing 
terms/? and q, may then be calculated by means of the influence 
functions presented earlier. The results take a particularly simple 
form if the integrals are evaluated at r = 2mr, m an integer, 

0 <> m < n, namely 

5u t (2 mn) = 22F* 2 mn 

<5 (2 mn)=F* 2 (68 mV - 80 mn) 


where and are submatrix elements of the complete state 
transition matrix [eqn(19)]. Eqns (32), (36a), and ( 37 a) then 
provide the open loop solution for the steering angle correc- 
tions, viz: 


Viny jz* inn, j - 


<Sa T =(cost, sin 1,1} A 1 (r 0 ) / <2> x (t 0 ) 


T 0 <?<T f 


d Pz= {cost, sin i}B 1 (t 0 ) K (t 0 ) 


T'O ^ T ■“C X f 


In order to obtain the closed loop linear feedback control 
solution mentioned earlier, it is only necessary to replace r„ by r 
in eqns (38) and (39), which are then applicable for 0 < t <y. 


8s i (2mn)=F* 2 (36 it+8 mV) ( 41 ) 

If these are employed as corrections to the nominal values with 
2 mn: replaced by t, the result is incorrect by the omission of 
certain oscillatory effects; however, it is thought that these will 
e unimportant in comparison with the secular terms which are 
properly accounted for. In any event, the idea is to provide the 
closed loop control system with some anticipation for errors 
accruing on account of non-linearity. 

One of the most significant limitations arising from the 
approximations made m the foregoing analysis is the restriction 
that the change m arrival time from the nominal be small 
compared with the orbital period. Shift in arrival time is 
direcdy proportional to the difference between actual and 

m va ues the lmearized ener sy parameter, a + 2 V . 
Furthermore, analysis reveals that relatively large steering 
corrections are associated with errors in this variable. Since 

tion n 7 t gy pa + rai 7 te r IS a monotOT icalIy increasing func- 
ion of time, it therefore appears reasonable to consider this 
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parameter as a candidate for independent variable in the 
mechanization of the system. According to this scheme, the 
system gains and the state variables along the nominal would be 
generated (or stored) as functions of energy-to-be-gained, the 
difference between the terminal and instantaneous values of 
v + 2rj. In an analysis conducted with an independent variable 
having fixed terminal value, complexities arising from shifts in 
terminal time would be avoided and there would be one less 
approximation required. 

There is another point to which scant attention has been 
paid in the present investigation, which is essentially a feasibility 
study, and this is the choice of reference orbit and the position 
of the target vehicle in relation to the reference axis system. 
We have tacitly assumed the target to be moving in a perfectly 
circular orbit and chosen our axis system location for analytical 
convenience as being at the initial point of the nominal orbit 
transfer trajectory which terminates at the target. It seems clear 
for a number of reasons that the appropriate choice in actual 
implementation of a system would be a reference circular orbit 
having the period of the target’s orbit and a reference axis 
system moving along this circular orbit in the vicinity of the 
target. One advantage of such a choice is the facilitation of 
guidance in terms of relative position and velocity measurements. 

Finally, it is pointed out that by regarding F* as an auxiliary 
state variable (as opposed to a parameter) defined by the differen¬ 
tial equation, F* f = 0, it then becomes possible, with little real 


increase in complexity, to derive an additional term for the 
guidance law whose function would be to counteract the effects 
of instantaneous fluctuations in the reduced thrust-acceleration 
level. The procedure for treating system parameters as auxiliary 
state variables is discussed at greater length by Kelley 1 . 

Conclusions 

The low-thrust quasi-circular orbital rendezvous example, 
presented in this paper, illustrates a possible practical applica¬ 
tion of an optimal guidance scheme developed in a previous 
publication. The closed-form result obtained for guidance correc¬ 
tions should be safely applicable for correctional manoeuvres 
during the last several revolutions of a low-thrust ascending 
(or descending) approach to rendezvous. Some numerical 
computations designed to establish the range of validity of the 
guidance approximation are currently in progress. 
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Appendix 

The matrices appearing in eqns (36) through (39) are: 


J = 


1 


-1 

0 

0 


0 0 

0 -1 

0 0 


[i;] 


A 


(2 nn—t 0 ) (sin2r 0 ) 
2 4 


J 


(1 —cos2t 0 ) 


(an 2 To )_ p nn _ Tq -j _ 2 sin Tq 


(1 — cos 2 t 0 ) 

‘ 4 

( 2 nn-T 0 ) (sin 2 r 0 ) 
2+4 

(1 —COSTo) 2 




B = 


2 nn — t 0 sin 2 1 0 


1 — cos2t 0 
— 4~“ 




1 —cos2t 0 
4 

cos t 0 
2 sin t 0 
— sin t o 


2n7C — r 0 j sin2T 0 


— 2sinr 0 

— (3— 4 cost 0 ) 
2(1 —cos t 0 ) 


3 sinr 0 

— 6(1— cos t 0 ) 

(4 —3 cost 0 ) 


— sin t 0 
(1 —cost 0 ) 

(4 nn — 2 r 0 + 2 sin t 0 ) 


-2(1 —cost 0 ) — (6 — 3 t 0 + 4 sin T 0 )-(12n7i-6T 0 + sinT 0 ) 
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< 2>2 = 


COS T 0 

— sin t 0 


also if 

2 mi —To 
x= -- 


2 


, 0<x<nn 


sin Tq 
cos t 0 


det M| -~4x 3 —4sin 2 x(3--sin 2 x)x + 8 sin 3 xcosx 7^0 

for x > 0 

del \B\ = .x 2 - ~ (sin 2 2 x) #0 for x>0 
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Summary 

A synthesis procedure for a wide class of optimal processes is presented. 
Linear processes are considered under the following types of optimiza¬ 
tion criteria and constraints: (1) Under control amplitude constraint, 
either minimize control effort or minimize time. (2) Under effort 
constraint, minimize time. (3) Under both amplitude and effort con¬ 
straints, minimize time. (4) Under no constraint, minimize effort. In 
each case, the control system is to take the state vector from a given 
initial state to a given terminal state, or to a given terminal closed 
convex set. Control effort is understood either in the sense of maximum 
amplitude or of an integral of a certain function of the control. 

It is well known that such optimal controls can be given in terms 
of a solution of the adjoint system of differential equations of the 
original system. The main difficulty in computing the optimal control 
lies in finding the initial conditions of the adjoint system. By geometric 
arguments, a function of the initial condition is developed, and it is 
shown that this function attains a maximum at the desired value of 
the initial condition. Furthermore, the gradient of this function is 
shown to have a simple form, so that the maximum can be directly 
computed by the method of steepest ascent. 

Sommaire 

Une procedure de synthese pour une large classe de processus opti- 
maux est presentee. Des processus lin^aires sont envisages avec les 
types suivants de criteres d’optimalisation et de limitations: 1) Avec 
la limitation de Tamplitude de commande, soit la minimaiisation 
de Teffort de commande, soit la minimaiisation du temps. 2) Avec la 
limitation de Teffort, minimaiisation du temps. 3) Avec la limitation 
aussi bien de Tamplitude que de Teffort, minimaiisation du temps. 
4) Sans aucune limitation, minimaiisation de Teffort. Dans chaque cas 
le systeme de commande doit amener le vecteur d’etat a partir d’un 
etat initial donne jusqu’a un etat final donne ou bien jusqu a un 
ensemble convexe ferm6 final donn6. L’effort de commande est com- 
pris dans le sens soit de Tamplitude maximum soit de l’intdgrale d’une 
certaine fonction de la commande. 

II est bien connu que de telles commandes optimales peuvent etre 
donnees sous forme d’une solution du systeme adjoint d’6quations 
differentielles du systeme original. La principal difficult de calcul de 
la commande optimale reside dans la determination des conditions 
initiales du systeme adjoint. A l’aide d’arguments g6ometriques une 
fonction des conditions initiales est etablie et il est montre que cette 
fonction atteint un maximum pour la valeur d6siree des conditions 
initiales. II est montre, ensuite, que le gradient de cette fonction pos- 
sede une forme simple, de sorte que le maximum peut 6tre directement 
calcule par la methode de la pente la plus rapide. 

Zusammenfassung 

Ein Syntheseverfahren wird erlautert, welches fur eine groBe Klasse 
optimaler Prozesse giiltig ist. Lineare Prozesse werden nach folgenden 
Gesichtspunkten der Optimierungskriterien und Beschrankungen 
untersucht: 1. Bei Begrenzung der Amplitude der StellgroBe mache 
man entweder die Regelleistung oder die Zeit zu einem Minimum; 

2. bei Begrenzung in der Leistung suche man das Minimum der Zeit; 

3. liegen Begrenzungen in der Amplitude und in der Leistung vor, 
mache man die Zeit zum Minimum; 4. sind keinerlei Begrenzungen 
vorhanden, suche man das Minimum der Leistung. 


In jedem Fall hat das System den Zustandsvektor von einem ge- 
gebenen Anfangszustand bis zu einem bestimmten Endzustand zu 
durchlaufen, oder bis zu einer bestimmten geschlossenen, konvexen 
Endflache. Die Regelleistung wird im Sinne eines Scheitelwertes oder 
eines Integrales fiber eine Funktion der RegelgroBe verstanden. 

Bekanntlich konnen derartige optimale Regelungen als Losung 
des adjungierten Systems der Differentialgleichungen des xirsprung- 
lichen Systems angeschrieben werden. Die Hauptschwierigkeit bei der 
Berechnung der optimalen Regelung besteht im Auffinden der An- 
fangsbedingungen des adjungierten Systems. Hier wird nach geome- 
trischen Uberlegungen eine Funktion der Anfangsbedingung ent- 
wickelt; man kann zeigen, daB diese Funktion bei dem gewfinschten 
Wert der Anfangsbedingung ein Maximum erreicht. Ferner ergibt 
sich, daB der Gradient dieser Funktion eine einfache Gestalt hat, so 
daB* das Maximum unmittelbar nach der Methode des „steilsten 
Anstieges“ berechenbar ist. 


Introduction 

The structure of optimal control processes is the subject of a 
great deal of current research. The maximum principle first 
formulated by Pontryagin, Boltyanskii, and Gamkrelidze 1 has 
been one of the fundamental results. This principle makes it 
possible to find optimal control functions for a wide class of 
optimal processes in terms of a solution of a linear differential 
equation which is related to the differential equation which 
describes the given control system. A difficulty arises, however, 
if one wishes actually to compute the optimal control, because 
the maximum principle does not prescribe the initial conditions 
necessary to solve the auxiliary differential equation. This paper 
presents an iterative procedure for computing these initial 
values for certain particular problems. 

The method which is set forth is applicable to systems which 
can be described by a linear system of ordinary differential 
equations of the form: 

i = A(t)x + B(t)u(t) (1) 

where x is the state vector (say of dimension ri) in the phase 
space of the system, u ( t ) is the control vector (say of dimension 
r), and A and B are n x n and n X r matrices which are con¬ 
tinuous in the time t. 

Suppose that a ‘control region’ U in r space is given, and that 
the control functions u (t) are constrained to take on their values 
in U. Vector functions with range in U 9 each of whose com¬ 
ponents are measurable, will be called admissible controls. 
If x (t) is a solution of eqn (1) with x (0) = x Q and u (t) = u * (t), 
and if x (t') = x x for some t' > 0, it will be said that u* (t) 
transfers x 0 to x 1 in time t'. 

Finally, suppose that a scalar valued function (j) 0)> whose 
domain contains U, is given. <j) will be called the effort function , 
and if u (t) is an admissible control function, the functional 

T 

(j) (u (t))dt=e(u(t)) 
o 
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(T represents the duration of the control process) will be referred 
to as the control effort associated with u it). Physically, s may 
represent fuel or gas consumption, eletrical power loss, etc. 

Given two points x Q and x x in phase space, we shall consider 
the following optimization problems: (1) find an admissible 
control which transfers x 0 to x 1 in minimum time; (2) find an 
admissible control subject to the constraint e (u (t)) < M 
(where M is a given constant) which transfers x 0 to x 1 in minimum 
time; (3) given a time T> 0, find an admissible control which 
transfers x 0 to x 1 in time T and which in so doing minimizes the 
effort s ( u (0). In each case we shall have to assume that 
B , U 9 and (j) satisfy certain conditions. These, however, 
include a number of important physical problems. 

According to the maximum principle, the optimal control for 
each problem is given in terms of a solution of the equation 

i(2) 

(where A* is the adjoint matrix of A). To find the proper initial 
conditions for eqn (2), we shall define a real valued function 
F(rj) = F(7]\..., rf) such that F attains its maximum at those 
values of rj for which the solution of eqn (2), with \p (0) = — rj, 
determines the optimal control. We shall also give an iterative 
procedure for computing this maximizing value. The method of 
computation is particularly suitable for hybrid analogue digital 
computers. 

The synthesis procedure described in this paper exploits one 
of the prime features of the maximum principle: the optimal 
control is determined by n (or n + 1) real numbers—the initial 
values of the adjoint system. Our results are an extension of a 
procedure previously developed for the so-called ‘bang-bang’, 
time-optimal problem 2 . 


I, Time-Optimal Controls with no Effort Constraint 


The time-optimal control problem with no effort constraint 
will, in general, have a solution only if U is compact. It will be 
assumed that U is compact and convex, and contains the origin 
as an interior point. First consider the case with x x = 0. This 
is physically the most important case. 

The general solution of eqn (1) with initial value x (0) = x 0 
is given by 


x(t) = X(t) 



X 1 ( 5 ) B (s) u (s) ds 


(3) 


where X ( t ) is the n x n matrix function satisfying the equations 


X(t)=A(t)X(t); 

Let 


X(0)=I (the identity) 


C(0= - 


X 1 (s)B (, s) u (s) ds: u (s) admissible i (4) 


It is clear that C (t) consists of those points which can be trans¬ 
ferred to 0 in time using an admissible control. Because of the 
hypotheses on U, it follows immediatelly that C ( t ) is bounded, 
convex, contains the origin, and that C(t') 3 C(t) for t' > t 
ft can be shown that C(t) is closed (a proof for the special case 
ot U a polyhedron is given by Pontryagin et al A, and this proof 
is easily extended to U any compact, convex set). Finally, it 
follows from the compactness of U that if xeC(t) for all 
f > t , then xeC (f'). It then immediately follows that there is 
a smallest t* for which * 0 eC((‘) ; i. e „ there is a time-optimal 


control which transfers x Q to 0 in the minimum time t ! \ Further¬ 
more, x 0 is a boundary point of C(t*). (See Reference 4, Lemma 3 . 
It is easily seen that the hypotheses of this lemma are satisfied 
in the present case.) Therefore, there is a support plane to 
C (t*) at x 0 . Let 97 * = O 7 * 1 , ..?y* n ) be the vector normal to this 
plane directed away from (?(/*), so that 


i. e., if 


then 


77**x 0 >?7**x all xeC(t *) 


*n= — 


X -1 (0 B(t) u*(t)dt 


ft* _ (V 

I -I7*-X *0)5(0-^•X" I 0)B(0«0)df 

(5) 

for every admissible u (/). Note that the vector function 
— rj* * X^it) satisfies eqn ( 2 ) with initial value \/j ( 0 ) 

(The geometric arguments of this section were first given by 
Bellman, et al . 3 for a particular case of this problem, and have 
since been used by various other authors.) 

One further assumption is now made concerning A , B , and U. 
It is supposed that if \j/ (t) is any non-trivial solution of eqn (2), 
the function [\jj (f) B (/)] * v of the vector variable v has a unique 
maximum in U for almost every t, 0 <r t < 00 . (Since the origin 
is an interior point of U s this maximum is positive.) This con¬ 
dition is referred to here as the unique maximum condition. 
If U is the cube given by | u j | < 1 , / = 1 ,..#■ [where u == (u\ 
•••, “ r )L this condition corresponds to the concept of normal 
control systems introduced by La Salle 4 . If U is an arbitrary, 
convex, closed polyhedron, this condition is equivalent to the 
general position condition described by Pontryagin et al A 

The following notations are now introduced: Let \// (/, rj) be 
the solution of eqn (2) with initial value 1 // (0) — — ?j. Let 
u (t, rj) be the admissible control function which has the prop¬ 
erty that 

DA 0, rj) B (0] * u (t, rj) = max [ 1 J/ (r, rj) B (*)] * v (6) 

veU 

By hypothesis, u (t, rj) is uniquely defined for almost all t > 0. 
(We shall disregard sets of measure zero when describing 
control functions.) 

It follows from eqn (5) that u*(t) - u (t, r h ). (This statement 
is a special case of the Pontryagin maximum principle.) Thus, 
if r /* is known, the solution ij/ ( t , rj#) can be computed from 
eqn ( 2 ), as a result of which the optimal control u*(t) is uniquely 
determined by the maximum principle. 

A method for finding 97 ^ is now given. Let 

z (t, rj) = - X~ 1 (s) B(s) u ( 5 , rj) ds (7) 

for any non-zero vector r\ and any / > 0; z ( t , rj) e C (t). By 
definition of u (t, rj), and by the unique maximum condition, 
it follows that 


ri'z(t,rj)>rj-lj for all £eC(t), C=^z(t,rj) (8) 
Consider the function 


f(t,fi;x 0 ) = n-lz(t,rj)-x 0 ] ( 9 ) 
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For fixed ?y # 0, / is a continuous, strictly monotonically 
increasing function of t (since the maximum given by eqn (6) 
is positive). Let the domain of/be restricted to those ?y for which 
7 ] * * 0 = —/(0, r};x Q )> 0. If 2 (t* 9 rj) # x 0 ,f(t*, rj; x 0 ) > 0 
by eqn (8), and, consequently, /(/, rj; x 0 ) = 0 for some unique 
* e (0, f*). On the other hand, if z (/*, rj) = then/(/*, 97; x 0 ) 

= 0 and /(f, r\\ x 0 ) < 0 for t < t*. Let F(rj ; x 0 ) be the value 
of t which satisfies the equation 

f(F(tr,x 0 ),ri;x 0 ) = Q (10) 


Clearly, C (?) consists of the points (x 1 ,. x n , x n+1 ) such that 
the point (a 1 , ..., x w ) can be transferred to the origin in time t 
(using an admissible control) with control effort x n+1 . Note 
that C 00 =3 C 0) if t' > t. 

Two further assumptions concerning A, B, <5 h, and U will be 
made. First it is assumed that C ( t ) (for every / > 0) is closed and 
convex and has the property that if x e C (t) for all r > t, then 
xeC 0); and secondly, it is assumed that a generalized unique 
maximum condition is satisfied: for every non-trivial solution 
\j/ (t) of eqn (2), and every non-positive constant ft the function 


Then F is defined, and 0 < F(rj; x 0 ) < /*. Furthermore, 
F(?y; * 0 ) = t* if, and only if, z(t* 9 rj)=x 0 in which case 
[see eqns (3) and (7)] u (t, rj) is the time-optimal control which 
transfers x 0 to 0 in time t*. Note that the vector rj which deter¬ 
mines the optimal control is not unique. Nevertheless, the 
optimal control is unique almost everywhere. 

In summary the following has been shown: given a control 
system described by eqn (1), a convex, compact control region U 
which contains 0 as an interior point, and an initial point in 
phase space x 0 . If there is an admissible control which transfers 
x Q to 0, and if the unique maximum condition is satisfied, then 
there is a unique time-optimal control u (/, rj) defined by eqn (6), 
where \j/ (/, rj) is the solution of eqn (2) with 1 j/ (0) = — rj and 
7 ] is any vector for which the function F defined by eqns (7), (9), 
and (10) takes on its maximum value t*. 

Now suppose that x 1 is not the origin. In fact it can be 
supposed that the Target point’ is a (continuous) function of 
time f (t). In addition, it can be supposed that the right-hand 
side of eqn (1) has an additional (continuous) ‘forcing term’ h it), 
and remove the restriction that 0 e U. Then, it can still be 
shown that if there is an admissible control which transfers 
to £ (t) in some time t, then there is a time-optimal control of 
the form u ( t, rj), where ?y has an analogous geometric inter¬ 
pretation. An analogue of the function F can then be con¬ 
structed ; but in general, F will only be defined in a neighbour¬ 
hood of the optimal values of 77. However, F will again have a 
maximum at these (and only these) optimal values of rj. For a 
more detailed discussion see Neustadt 2 . 


II. Time-Optimal Controls with Effort Constraint 


Now consider the time-optimal problem in the presence of 
effort constraint. The discussion will be confined to the case 
where the ‘target’ x 1 is the origin; the extension to the general 
case can be carried out as indicated above. 

In this section it need not be assumed that U is bounded (in 
fact, it may be the entire r dimensional space, in which case there 
is no constraint on the allowed values of the control). It will be 
supposed, however, that U is closed and convex and contains 
the origin. The effort function (j) (u) will be assumed to be 
continuous, bounded from below on U, and to satisfy the 
relation (j) (0) == 0. Further implicit restrictions on 0 and U 
are stated below. 

Corresponding to the set C (t) of Section I, the set C (t) in 
(n + l)-space will be defined as follows 


com - 


X 1 (s)B (s) u (,s) d s, 4> (u (s)) d s 


: u ( s ) admissible f 


of the vector variable v has a unique maximum in U for almost 
every t, 0 < t < 00. (Since (j) (0) = 0, and 0 e U 9 this maximum 
is non-negative.) 

Three important cases for which the first assumption is 
valid are the following: 

(1) If U is the unit cube | u j | < 1, j = 1, ..., r, and <j> («) is 

a non-negative convex function which is defined (and continuous) 
in an open set which contains U, and which takes on its maximum 
value in U at each vertex of U , C ( t) satisfies the above hypo¬ 
theses 5 . In particular, functions (j> (u) of the form X 1 \ | 3) , 

7=1 

where the Aj and (p—1) are non-negative constants, fall under 
this category. 

(2) If U is the unit sphere X r (« 3 ) 2 = ||w|| 2 < 1, and 

7=1 

qi > (u) = ||«||, the set C (t) has the desired properties. 

(3) If U is the entire space, and cj > ( u ) = X T j=1 Aj \ u 3 \‘ p where 
the % and {p — 1) are positive, the first assumption is also valid. 

The generalized unique maximum condition is more difficult 
to verify in particular examples. In general, this condition will 
be violated only if the time-optimal control is not unique. 
A more detailed discussion is given by Neustadt 5 . 

Corresponding to the function u ( [t , rj), the function u, (t, rj) 
will be defined, where r\ is any (n + l)-vector (ry 1 ,..., r/, 
1 ) = (rj, ?y n+1 ) with 97 # 0 and '/y w+1 < 0. Namely, let u be 
the admissible control function which has the property that 

[*A (t 9 rj) B (0] ■ u (t, rj) + rf l +1 (j) (u (t 9 tj)) 

= max [\ff ( t 9 rj ) B(t)v + r} n + 1 (p (r)] (11) 

veil 

Here xj/ it, rj) is defined as in Section I, and, by hypothesis, 
u (t, rj) is uniquely defined for almost all t > 0. 

Now let l be the line segment in (n + l)-space defined by 

l = {x=(x 0 ,O:t;<M} 

where x 0 is the given initial position in phase space, and M is 
the maximum allowed value of the control effort. Clearly, there 
is an admissible control which transfers x 0 to 0 in time t with 
control effort not exceeding M if, and only if, C U) meets l. 
Thus, one is looking for the smallest value of t for which C(t) 
meets l. Because of the assumptions on C ( t ) and (f) («), it follows 
that there is such a smallest t ; i. e., there is a time-optimal control 
satisfying the effort constraint. Denote the minimum time by t*. 
It can also be shown that because each C (?) is convex, every 
point of C ( t *) n / is on the boundary of C (/*). The generalized 
unique maximum condition implies that every boundary point 
of C ( t *) is an extremal point of the set. Hence, there is only one 
point x* = (x 0 , |) e / fl C (?*). Now there are two possible 
cases : either £ = M or £ < M. If £ < M, C (t*) meets the line 
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through (x 0 , 0), parallel with the x n+1 axis, only at x*, and C (/), 
for t < t*, does not meet this line at all. Thus, t* is the minimal 
transfer time from x 0 to 0 in the absence of the effort constraint. 
The problem then reduces to that considered in Section I. 
Henceforth it is assumed that f = M, so that x* = (x 0 , M). 

Clearly, x* is on the boundary of C (/*), and there is a 
support plane to C (/*) atx*. Let the normal to this plane that 
is directed away from C (r*), be 17* = ( v *\ ..., = (^0). 

It is clear from the geometry that /3 < 0 . As in Section I, it 
follows that 

x*=(x 0 ,M ) 


“ (/, tj) is the time-optimal control which transfers x 0 to 0 in 
time t* with effort M. Again, the vector tj which determines the 
optimal control is not unique, although the optimal control 
is unique a. e. 

Thus, summarizing: given a control system described by 
eqn ( 1 ); a convex, closed control region U which contains the 
origin; a real valued, continuous function (j> (u) defined on U. 
which is bounded from below and takes on the value 0 at u =0; 
a control effort constraint of the form J 0 * <j> (u(s)) dr < M; and 
an initial point x 0 in phase space. If ( 1 ) the generalized unique 
maximum condition is satisfied; (2) the sets C (t ) are closed and 


X 1 (t)B(t) u(t, f/*)df, J 4 >(u(t,t]^))dtJ 

and u(t,i 7*) is the desired optimal control. Since this control 
is determined by the solution i J/ (t, v *) and /$ [see eqn (11)], and 
since these quantities are in turn determined by 77*, the optimal 
control can be computed once iy* is known. 


convex; and ( 3 ) xeC (r) for all r > t implies that xeC (t), 
then, if there is an admissible control satisfying the effort 
constraint which transfers Xq to 0, there is also a unique time- 
optimal control u (/, 77) defined by eqn (11), where t] = (rj t ij n+1 ), 
and i/r (t, r t ) is the solution of eqn (2) with 0- (0) = — ??,’ and 17 
is any vector which maximizes the function F(tj;x 0 ) defined as 
the smallest value of t for which/ (t, 17; * 0 ) [see eqn ( 13 )] vanishes. 


As in Section I, a function is defined which takes on its 
maximum at the desired values of jy*. 

Let 


III. Minimum Control Effort Optimizations 


z (t>rt—(^ j 1 (s) B (s) u(s, rj) dr, <f>(u(s,ti'))dsj 

for every vector tj = (rj, ?f+ 1 ) for which rj # 0 and r/ n+l < 0 
Since u is admissible, z (r, rj) e C (r). By virtue of the generalized 
unique maximum condition 

tfz(t,ti)>trC for every £eC(t), £=£z(t,ij) ( 12 ) 

Consider the function 

f ( t,ti)-x *] 

= J o bp (S, n) B (s) u (s, 17) + rf -+ 1 <f> (u (s, ,))] d s- r x* ( 13 ) 

where we only consider those r\ = (77, with 7f +1 ^ 0 and 
^7 7^ 0. Since the integrand in the above expression is non¬ 
negative, / is a continuous, non-decreasing function of t (for 
fixed rj), Let^us restrict the domain of / to those ij for which 
u X * = ,7/ (0 ’ V;Xo) > °-I fz ( - t *’ ^ ^*Mhen/(r*,i7;.r o )>0 
y m q L (1) ’ and consec l uentI y> f(F>i;x 0 ) =0 for some 
t e ( 0 , t ), On the other hand, suppose that z(t* 9 n)=x *. 

Then, clearly, / (t*, t];x Q ) = 0. Furthermore, it can be shown 
that in this case/(r, tj; x 0 ) < 0 for t < t*. Indeed, suppose that 
f( ( ,y;x 0 ) =f(t*, ri\x a ) = 0 , where t‘ < t*. Then, n ■ z (t’ n) 

Ft* 

J ^ [ip (s, ri)B (s) u(s, v) + ri n+1 4> {u (s, 17))] ds = 0 


Now consider the problem in which two points, a-„ and x lt 
in phase space, as well as a time T > 0, are given, and it is 
desired to find an admissible control it (t) which transfers 
*o t0 A 'i in time T while minimizing the control effort s («(r)). 
Assume that (1) U is closed and convex; (2) cj) («) is continuous 
and bounded from below on U; ( 3 ) C (T) is closed and convex; 
and ( 4 ) the generalized unique maximum condition is satisfied. 
Finally, assume that the optimization problem is not vacuous, 
i. e., that there exist admissible controls which transfer x 0 to rq 
in time 7 ) and that there is more than one possible control effort 
for such controls. 

It follows from ( 3 ) that there is an admissible control u (/) 
which transfers x 0 to x 1 in time T with effort A if, and only if, 
O', X) e C (T) where y = x 0 — (T) x v Since 0 is bounded 

from below and C ( T ) is closed, there is a point y* = (y, A*) e 
C ( 7 ) such that (y. A) 6 C ( T ) implies that A > A*; i.e., there 
exists an optimal control (with minimum effort A*). Clearly, 
y is on the boundary of C ( 7 ), and, by hypothesis, C (7) 
contains points (y. A) with A > A*. Since y * is an extremal 
point of C ( 7 ), there is a support plane to C (7) at y* with 
normal 17* = (j;*, r ^ +1 ), where < 0. It follows as in 
Section II that y* = z ( 7 ,77*), and that u (t, 17*) is the unique 
optimal control. 

We shall again construct a function of tj which takes on its 
maximum at those values 17* for which u ( t , 17*) is the optimal 
control. Without loss of generality, we shall confine ourselves 
to those vectors 77 for which ■>'i n+1 =_l. 

I f V = (»?, - 1 ), let 


But u (t, 77) is defined by eqn ( 1 1), and this maximum (which is 
the integrand above) is non-negative, and therefore zero in 
(t, t*). Since cp (0) = 0, u (/, 17) = 0 for t' < t < t* by virtue 
of the generalized unique maximum condition. Thus, z ( t ', 17) = 
z(t*, V)= x*, which implies that u (r, 77) is an admissible 
control which transfers a 0 to 0 in time t' < t* with effort M 
contradicting the definition of t*. 

Now let F{iy, x 0 ) be the smallest value of t for which / (t, 
1 V’ x o)~ 0 . Then V is defined, and 0 < F(tj; x 0 ) < t*. Further¬ 
more, F (17; x 0 ) = t* if, and only if, z ( t *, >7) = x*, in which case 


g(rt = ri-y-tl-z(T,ti) ( 14 ) 

If z ( 7 , 17) # y*, then 

girt—try—X*—tj-z{T,ti)+x* 
=n-y*-rz{T,tj)+x*<x* 

by eqn (12). On the other hand, if z (7,17) = y *, then A* = 

Thus, g (rj) < X*, and g (rj) = A* if, and only if, z (7, ij) = y* 
in which case u(t, tj) is the optimal control which transfer x 0 
to in time T with minimum effort 
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To summarize: given a control system described by eqn(1), 
and a time T> 0; a convex, closed control region U; a real 
valued, continuous function (j> ( u ) which is defined on U and is 
bounded from below; control effort defined by Jq (j) («(/)) d t; 
and an initial point x 0 and a target point x x in phase space. 
If the generalized unique maximum condition is satisfied, and 
if C ( T) is closed and convex, then (provided the optimization 
problem is not vacuous) there is a unique (minimum effort) 
optimal control u (t , tj) defined by eqn (11), where ij = (r/ — 1) 
and \j/ (t , rj) is the solution of eqn (2) with \jj (0) — — r\, where rj 
is any vector which maximizes the function g (rj) given by 
eqn (14). Furthermore, the maximum of g is the minimum 
effort (j) ( u(t , tjj) d t. 


IV. Computing the Optimal Control 

An iterative procedure for finding the maxima of the func¬ 
tions F, F, and g will now be described. 

First consider the problem described in Section I. Let 
(the initial estimate for the maximizing rj) be any vector such 
that t] q * x 0 > 0. Then, if r]j is the estimate from the Jth iteration, 
choose rj j+1 to be 

rij +1 = V j ~K\_z (F (t]j; x 0 ), ijj ) - *o] I l5 ) 

where K is a sufficiently small positive constant. In fact, 
it will be shown that F(% +1 ) > F(r]j) (provided that K 
is sufficiently small and F ( rjj ) < f *), and that — ly has the 
direction of grad F(rjj) if Fis differentiable (in which case our 
iteration method becomes that of steepest ascent). The argument 
x 0 in the functions / and F will henceforth be dropped for ease 
of notation. 

Consider /( t , rj) as a function of rj with t fixed. It is shown 
in Section V that 

Pi 

— : [f yz (t, rff] = z l (t, rj), the ith coordinate of z (t, rj) 
drj 1 

for every t > 0. Since d[ry(— x 0 )]/^[rj l = — x 0 \ it follows 
from eqn (9) that 

grad / (t, rj) = z(t,ij) — x 0 (16) 

If Fir},) < t*, z (F(%), rjf) # x 0 (by definition of t*), and grad 
/ (F(rjj), rjj) # 0. By definition, /(F(%), r h ) = 0, and, therefore, 
by well-known properties of the gradient, 

/ (F (rij), r\j - K grad / (F (rij), tjj)) < 0 

if K > 0 is small enough. Hence, by definition of F, 


F (rjj - K grad / (F (rjj), rij)) > F (rjj) 
or (see eqns (16) and (15)), 

F(Vj+i)>F(rij ) 

It is shown in Section V that z (t , rj) is continuous in rj. Also, 


Q/ = a_ 

dt dt 


i J/ (s, rj) B (s) u (s, rj) ds 


= \J/(t,tj)B(t)u(t,rj) 


= max t j/ (t , rj) B ( t ) v 

veU 

is easily seen to be continuous in t and rj. Hence, if bf/bt A 0 
[and since 0 is an interior point of U, bfjbt — 0 only if i j/ (t, rf). 


B{t) = 0] for t = F('!),) and rj = %, then, by virtue of the 
implicit function theorem, F(ij) has continuous derivatives at 
rj = rjj which are given by 


8 F (if.,) 9/ (F (r,j), rjj) 1 9/ (F (rjjX ) 


drj‘ 


or, 


grad F (rj 


»-0 


drj 1 

9/ (F (jj j)j Vj) 


dt 


dt 


-l 


grad f(F(tjj), rjj) 


."^T 1 

Jdt_ 


[z (F(ijj), rjj) - x 0 ] 


(17) 


Thus, the iteration described by eqn (15) is indeed in the direction 
of ‘steepest ascent’. 

It is clear from eqn (17) that F(r/) has no local extrema at 
values of 77 other than those for which 2 (F(^), rj) — x Q , and at 
these values F(rj) = r* and u (t, rj) is the time-optimal control. 
Hence, if the iteration method converges, it must converge to a 
value of rj such that u (t , rj) is the time-optimal control. 

The computation procedure is particularly adaptable to 
hybrid analogue digital computers. Typically, it might proceed 
as follows. A guess for 77 is made, this value is stored, and — 7 / 
is used as an initial condition in eqn (2). An analogue computer 
is then used to compute \j/ (/, rj) as a function of time. Simul¬ 
taneously, u ( t , rj) can be computed with an analogue ‘maximiz¬ 
ing circuit’ which must continuously compute the maximum in U 
of a time varying linear function of u [as given by eqn (6)]. 
A description of such a circuit, if U is a polyhedron, is given by 
Pontryagin et al. 1 A more general computation procedure is 
described by Neustadt 6 . Simultaneously with u it, rj), z ( t , rj) is 
obtained from an analogue integration [see eqn (7)], where 
X~ x (t) is computed as the solution of the equation 


A(z- 1 ) = -x-M,z- 1 (0)=/ 

Finally, the dot product rj • [z (?, rj) — x 0 ] is computed as an 
increasing function of time, and the computation is halted when 
the value of this product is zero. The corresponding time is 
F (’rj). The vector [z (F (??), rj) - x 0 ] is then read out and mul¬ 
tiplied by an appropriate negative constant — K, and this value 
is added to the stored value of rj to yield the next approximation, 
at which point the next iteration is begun. 

The maximum of the function F (rj) of Section II can be 
found in an almost identical manner. The problem is somewhat 
more difficult than that described above because of the following 
factors: (1) the vector rj has in + 1) coordinates (rather than n)\ 
(2) whereas F ( tj) is continuous, F (if) may have discontinuities, 
and (3) the computation of u ( t , rf) from eqn (II) is complicated 
by the fact that the function (of v) to be maximized is non-linear. 

Finally, the maximum of g (rf) (which must be computed to 
find the optimal control described in Section III) can be found 
more directly than the maximum of F(rf). We show in Section V 
that 

—rj ■ z(T,t]) = z i (T,tf), the ith coordinate of z(T,if), 
drj 1 

i = l,...,n 

Further, b(rjy)lrf — y\ so that (see eqn (14)), 
grad g(rj)=y-z(T,ij) 
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where z (T, if) = (£ (T, if), z n+1 (T, if))-, i.e., 

1 T 






Also, note that, by eqn (3) and the definition of y. 


y - z (T, if)=X~ 1 (T) [x (T, if) - xf] 

where x (t, tf) is the solution of eqn ( 1 ) with u(t) — u (t, q). 

Thus, an iteration procedure based on the method of steepest 
ascent with 

Vj+i=fIj+KgT&dg(i]j) 

= y\j+KX~ 1 (T)[x(T, gj) —Xj] 

can be used to find the maximum of g. Here K is a sufficiently 
small positive constant, and tjj = (rj j9 - 1 ). 


V. Final Derivations and Generalizations 


It will be proved that z ( t , rj) and z (t, rj) are continuous 
functions of rj (or rj), and that 

— t (fi • z (*. n)) = 0, v), ~ (r* (t, iff)=z l (t, n) 

0/7 0/7 


These conclusions follow from general properties of convex sets. 
Indeed, suppose that C is any closed and convex set in an 
m dimensional vector space and that r) 0 is a vector in this space 
with the following property: there exists a neighbourhood N 
of 7] 0 such that for every rj e N, the support plane to C, whose 
normal (which is directed away from C) is rj, meets C at only 
one point. This point is denoted by z (rj). Then z {rj) is con¬ 
tinuous at rj = 7] 0 , and the scalar function <f> (rj) = rj • z (rj) has 
continuous derivatives at rj = rj 0 given by 


Q ty^=z l (rj 0 ) i the ith coordinates of z(rj 0 ); z = l 5 ...,m 


dr I l j 


(18) 


To prove that z (rj) is continuous at rj = r] 0 , let rj $ be a 
sequence of vectors such that lim^^ rjj = 7 ; 0 , lim^^ z (rjj) = f. 
Then, by definition of z (??), 

rij’z(rij)>nj'z(Yi 0 ) 


Passing to the limit as y~>Co, one obtains 


no<>no'z(no) 

But by hypothesis, this is only possible if £ = z(rj). This implies 
that z (rj) is continuous, provided it can be shown that the set 
of vectors z (rjj) is bounded for any sequence % which converges 
to But the latter statement is easily verified. 

Relation (18) will now be proved. If rj e N, 

<t> (n) - 4> 0 ? o )=n • z in) - *io • z Oio)=no • z in) - no • z (y 0 ) 

+ 0l-ri o)-z (r\ 0 ) + (ri-ii 0 )-[z(ri)-z (>/ 0 )] (19) 

Let 7] — r, f> = ( 0 ,..., 0 , A,- rj, 0 ,.. 0 ), where A*?? is the ith 
coordinate of ?] — Thus, \\rj — % || =\A i r 1 \, and (?? — ?? 0 ) • 
z ( r /o) = (Aj rj) 2 * Oy 0 ). Hence, it follows from eqn (19) that 








^lo' z ( r l)~ r Io' z (Vo)\ 


A;J7 


+ ||z(>?)-z(^o)[| 

( 20 ) 


But '% ■ z (%) >?] 0 ■ z (rj), and i] ■ z (rj) > >/ ■ z (%), so that 


0 > rjo ■ z (rj) - 1 ] 0 ■ z (f/ 0 ) > bl - >/ 0 ]' [ z (»7o) - z (»?)] 
or. 


ho- z (f?)-flo' z 0lo)l s-Wl-VoW \\ z (jlo) z (^l)il 


|Afl| 

so that, by eqn ( 20 ), 


|Afl| 


:||z(/7)-z(/7o)|| 


(j>(rQ-4>{no) 

A;/7 




< 2 ||z(/ 7 )~z(/ 7 0 )|| 


Thus, eqn (18) follows from the continuity of z ( 77 ) at > 7 0 . 

Finally, mention is made of three problems which are related 
to the ones described in this paper, which can be treated by 
techniques similar to those described here. 

The first problem is a generalization in which the point x 1 
is replaced by an arbitrary compact, convex set. The second 
problem is the same as that of Section III, except that e (u (/)) 
is redefined by 

s (u ( t )) = sup max | u J (t) | 

0 <*<; T j 


This problem was first studied by Krasovskii 7 in his investigation 
of the time-optimal problem. The third problem is one con¬ 
sidered by Ho 8 in his investigation of the same problem. Namely, 
for given x 0 , x lt and T > 0, find an admissible control u (t) 
which brings Xq ‘as close as possible’ to x x in time T; i.e., find a 
control for which || x ( T ) — x x \\ is minimized, where x ( t ) is the 
solution of eqn ( 1 ). 

A detailed treatment of these problems will be given else¬ 
where. 


VI. Computational Studies 

A series of computational studies has been carried out to 
determine the general character of the iterative procedure. 
These studies were carried out for the time-optimal problem 
described in Section I. A hybrid analogue digital computer was 
employed to study second-order systems. An IBM 7090 digital 
computer was used in the investigation of a third-order system. 

A geometrical picture of the iteration procedure for a 
second-order system is helpful in interpreting the behaviour in 
the higher-order cases. Figure 1 shows a phase trajectory of 



Figure L Typical phase trajectory of z (t, rj) 
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^ (t, ?j) as a function of t for fixed rj. The point x Q is indicated 
together with a portion of the ‘plane’ (line) through x 0 , normal 
to rj. Figure 2 illustrates the procedure for obtaining the 
gradient for the steepest ascent corrections to rj from z ( t , rj) 
and x 0 . 

The initial tests were conducted for time-optimal regulation 
(xj = 0) of a second-order, undamped oscillatory system, with 
one control: 


■^T — •^'2 

± 2 = —C0 2 X i + ll ( t ), \u \< 1 


( 21 ) 


The initial direction of i] was taken opposite the state vector, 
i.e.. 


x 0 

ni llxoll 


The equations for z (/, rjj) were integrated on the analogue 
portion of the hybrid computer. The digital computer stopped 
the integration at the time t — F (yj) when /(*,??*; x, 0 ) — 0. 
The corrections to iji were obtained in the digital computer, fed 
to the analogue and the process repeated. The cycle continued 
until the terminal point of the z trajectory was within a specified 
distance of x 0 . 

The numbered sequence of z trajectories shown in Figure 3 



Figure 2. Illustration of gradient procedure 


converges to x Q in eight trials. The broken line represents the 
phase portrait of the system responding to the optimal control. 

A proper choice for the constant K in eqn(15) is very 
important. In the problem under discussion, the value K = 1 
proved to be satisfactory. However, it was possible to cause 
large oscillations about the correct solution by choosing K^> 1. 

The second case studied was time-optimal regulation of the 
system x x = x 2 , x 2 = u (t\ \ u | < 1 . 

A set of z (t , rj) trajectories for this system is shown in 
Figure 4. The support plane to C (/*) at x Q is almost parallel to 
the z (/, rj*) trajectory. In this case, a small change in rj produces 
a large displacement in z ( F(rj ), rj). Hence, it was necessary to 
programme K as a function of ||z (/, rj) — x 0 || in order to avoid 
large oscillations, and, on the other hand, to keep the number of 
iterations small. In the case shown, the value K = 0T proved 
satisfactory near the solution, but larger values were employed 
for large values of ||z (t, rj) — x Q \\. 

In addition to studies of time-optimal regulation, a number 
of tests on second-order systems were made to determine the 
behaviour when ‘pursuing’ moving target points. Satisfactory 
results were obtained in these tests also. 

The application of this technique to second-order systems 
presented no particular difficulty from the standpoint of stability 
of the iterations or finding a proper choice for K. The study of 
third-order systems, however, led to some interesting difficulties. 



Figure 3. Convergence of z trajectories, Case 1 



Figure 4. Convergence of z trajectories , Case 2 
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A simplified steering problem provided this example. The 
system equations are: 

c 


x 2 = x 3 

x 3 =— - u 2 — 30 
3 n-t 2 

(c an dju are given constants), and U is given by (uff + (w 2 ) 2 < 1. 
Here, also, x x # 0. 

This problem was solved entirely on a digital computer. 
Again, z (F(rj), rj) proved to be an extremely sensitive function 
of rj. As a result, it was necessary to determine the time F (rj) 
with great precison. Indeed, the maximum error which could 


be tolerated was of the order of 10~ 4 sec. This sensitivity is 
illustrated in Table 1 , whose values were chosen from a represent¬ 
ative case. This table presents F(rj) and the three components 
of [z (F(rj), rj) — x 0 ] for four representative values of rj . 

The extreme sensitivity of z (F(rj), ij) made it very difficult 
to converge to the correct value of rj, or correspondingly to 
bring z ( F(rj ), rj) near x 0 . In order to circumvent this difficulty 
the following two steps were taken. First, a carefully chosen 
programme for K was adopted. Secondly when successive values 
of F(rjj) differed by a preselected amount (corresponding to 
F{rjj) being very near the minimum time), a different com¬ 
putational scheme was used to obtain a precise estimate for ?/*. 
When the refinements were carried out, the convergence was 
rapid, with the number of trials generally less than 50. These 
computational aspects are discussed in more detail elsewhere. 


Table 1. Typical Variation ofFandz with rj 


Components of rj 

F(n) 

Components of z (F (rj), rj) — x 0 

0-54580027 0-01316405 1-0729829 
0-54580027 0-01316405 1-0729839 
0-54580027 0-01316405 1-0734830 
0-54580027 0-01314405 1-0734830 

148-56863 

148-56906 

148-56824 

148-56763 

i 

0-830 22,700 - 7-373 

0-397 52,705 -16-369 

1-259 - 7,410 1-640 

10-711 -121,610 31-768 


References 

1 Pontryagin, L. S., Boltyanskii, V. G., Gamkredlidze, R. V. and 

Mishchenko, E. F. The mathematical theory of optimal pro¬ 
cesses. (English transl.) 1962. New York; Interscience 

2 Neustadt, L. W. Synthesizing time optimal control systems. 

/. Math. Anal Applic. 1, No. 4 (1960), 484 

3 Bellman, R., Glicksberg, I. and Gross, O. On the ‘bang-bang’ 

control problem. Quart. Appl Math. 14, No. 1 (1956), 11 

4 LaSalle, J. P. The time optimal control problem. Contributions 

to the Theory of Nonlinear Oscillations, Vol. 5, pp. 1-24. 1960. 
Princeton; Princeton University Press 


5 Neustadt, L. W. Time optimal systems with position and integral 

limits. J. Math. Anal. Applic. 3, No. 3 (1961), 406 

6 Neustadt, L. W. Applications of linear and nonlinear program¬ 

ming techniques. Proc. Third Internat. Conf. Analog Computation. 
Opatija, Yugoslavia (1961) 197 

7 Krasovskij, N. N. On the theory of optimal control. Automat. 

Telemech., Moscow , 18, No. 11 (1957) 960 

8 Ho, Y. C. A successive approximation technique for optimal 

control systems subject to input saturation. J. Bas. Engng 84, Ser 
D, No. 1 (1962), 33 


DISCUSSION 


R. Sargent, Imperial College, London 

I was interested in Dr. Neustadt’s comment that the proof of convexity 
of the set C (t) in the paper could be extended to some non-linear 
systems. The usual proof depends heavily on the linearity of the equa¬ 
tions; but there is an additional difficulty if the equations are non¬ 
linear, since the set is defined in terms of the matrizant X (t), which, 
of course, is only defined for a linear system. One can still write the 
solution in the same form as eqns (3) for a non-linear system, but then 
X is also a function of the initial conditions, so that we no longer obtain 
a set with the properties of C (t), and it is not clear that the convexity 
proof provides the same type of solution as for the linear case. 

L. W. Neustadt, in reply 

Professor Sargent misunderstood my remarks concerning the convexity 
of the sets C ( t) under conditions less stringent than those described in 
our paper. 

The sets C ( t) are convex if the equations are non-linear in u but 
linear in x; i.e. if the control system is described by an equation of 
the form 

x{t) = A(t)x{t) + ^{ujt) 


where 0 is a continuous function, and if the set U of admissible values 
for the control variable u is compact (but not necessarily convex). 

The proof of the convexity and compactness of the sets C (t) under 
the above hypotheses is given in another paper 1 . 

Reference 

1 Neustadt, L. W. The existence of optimal controls in the absence 
of convexity conditions. J. Math. Anal Applications 7 (1963) 

A. R. M. Noton, Electrical Engineering Department, University of 
Nottingham, England 

In synthesizing optimal controls, the author has pointed out that the 
calculations reduce to the solution of differential equations with mixed 
boundary conditions. Because of the difficulties in coping with such 
boundary conditions, he reformulated the problem so that it becomes 
one of maximizing a function of the initial conditions xjj (0), eqn (2). 
Now, although the second-order system quoted, eqn (21), presented 
no difficulties, with the third-order system, the z function was extremely 
sensitive to variations in rj [== - ^(0)]. An entirely digital computa¬ 
tion then became necessary. 
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This difficulty should be compared with the experiences of West- 
cott, Florentin and Pearson, who adopted a more direct approach, 
and in their paper have suggested a procedure for coping with the 
sensitivity to variations in ip (0). The author has not. The benefits of 
his mathematical reformulation are not therefore evident. Perhaps he 
will discuss them. 

L. W. Neustadt, in reply 

The sensitivity of 2 (/, i\) to small changes in ?] is indeed a serious, but 
not insurmountable, problem in applying the computational method 
described in our paper. 

The method of circumventing this difficulty, as discussed by West- 
cott et ah , is confined to the free right-hand endpoint problem, whilst 
in our paper, the fixed right-hand endpoint problem is the one under 
consideration. In addition, the derivation given by Westcott et al. is 
a formal one, which can run into difficulties when the derivatives 
£>i/r/5 pi and bij/bpi fail to exist (and such cases are not difficult to run 
into). 

J. A. Stiles, Trinity College , Cambridge University , Cambridge , Eng¬ 
land 

Does the author consider that the results of Section I could be extended 
to simple non-normal systems? 

L. W. Neustadt, in reply 

The proofs in Section 1 lean rather heavily on the so-called unique 
maximum condition, or, in other words, on the normality of the control 
system. Whether the proofs can be extended to non-normal systems 
is open to conjecture. 

It is clear that certain difficulties will arise. Geometrically, non¬ 
normality generally implies that the sets C (t) have flat portions, i.e. 
that a support plane to C(t) meets C(t) at more than one point. For 
this reason, the geometric arguments given in Section V must be mod¬ 
ified. Analytically, non-normality means that the optimal control is 
not completely determined by the proper ijj (0). 

If the unique maximum condition is satisfied by the optimal control 
being sought, one should expect that the difficulties described in the 
preceding paragraph can be circumvented even in non-normal systems. 

R. Kulikowski, Polish Academy of Sciences, Warsaw U1. Koszykowa 
75/18 , Poland 

The problem considered in this paper belongs to the wide and im¬ 
portant class of multi-constrained control problems. A number of 
different analytical methods can be applied for the determination of 
the optimum control signal u ( t ), which is constrained and minimizes 
the given functional F [u]. A pretty general method may be based on 
the calculus of variations in an abstract space. In the case of strongly 
differentiable functionals F [u], the necessary condition for optimality 
follows from the known Lusternik theorem for conditional variational 
problems. The sufficient condition of optimality for continuous (or at 


least weakly semi-continuous) functionals F [u\, which are given in a 
region w of a weakly compact Banach space, can be determined by 
the so-called Weierstrass theorem 1 . When || grad F [it] 11 > 0, the opti¬ 
mum control n belongs to the boundary of w and the synthesis of the 
optimum control system can be reduced to the solution of a number 
of equations for the unknown switching instants. However, this method 
requires familiarity with some abstract concepts of non-linear function¬ 
al analysis, which are not very popular among engineers. A great 
advantage of the geometric arguments used by Dr. Neustadt and 
others in their papers is in the simplicity of the physical interpretation. 
However, the computational scheme used for determination of the 
unknown coefficients does not seem to be so simple. For example, in 
order to find the best time optimal control with amplitude constraints 
[u (t )] < 1 for the second-order system: x ± = x 2 , x 2 ~u (r), a number 
of trajectories should be constructed (as shown in Figure 4 ). On the 
other hand, using the known method, based on the construction of 
the switching curve in phase-plane, the optimum controller can be 
synthesized very easily. 

In connection with this problem it is interesting to know whether 
a faster convergence of the iterations (for coefficients) could be 
obtained if the K coefficient were determined from the condition 

^{^-Xgrad/[F(jj i ),'7i]} = 0 > i=1 > 2 

or chosen in another ‘optimal’ way. 

Reference 

1 Weinberg, M. M. Variational Methods of Investigation of Non¬ 
linear Operators (in Russian). 1956. Moscow; Gittl 

L. W. Neustadt, in reply 

Being unfamiliar with the calculus of variations in abstract spaces, 

I cannot comment in detail on the early part of Professor Kulikowski’s 
discussion. However, I should like to point out that the basic aim of 
our paper was to reduce the problem of finding an optimum function 
to that of maximizing a function of a finite number (equal to the order 
of the basic differential equation) of variables, since the latter seems 
so attractive computationally. Admittedly, even the second problem 
is not trivial. The methods of steepest ascent suggested in our paper is 
the most naive, and does result in computational difficulties. 

Since the writing of our paper, computer studies using other, more 
refined hill climbing techniques (in particular, a method described 
recently in the British Computer Journal by Powell) have been carried 
out, and time-optimal trajectories have been computed with relative 
ease. These results will be reported by Dr. Paiewonsky. 

As pointed out by Professor Kulikowski, the second-order ex¬ 
amples discussed in Section VI are easily synthesized by known switch¬ 
ing curves. However, in higher-order systems where switching surfaces 
are difficult to obtain and to simulate, or in systems where the optimal 
control can take on a large number of values, the method described 
in the paper should offer some strong advantages. 

With respect to the last of Professor Kulikowski’s points, Powell’s 
hill climbing method specifies the value of K in eqn (15). 
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Summary 

A general midcourse guidance problem is formulated as an optimal 
control problem. The problem consists of bringing a vehicle on to a 
nominal trajectory with a minimum expenditure of fuel. An amplitude 
constraint is placed on the thrust (or control) vector. The equations are 
linearized by considering perturbations about the nominal. 

The form of the optimal control follows from the maximum prin¬ 
ciple and is a vector function of the system adjoint solution. The 
optimal control is ‘on’ only when a scalar function of the system 
adjoint solution exceeds a fixed threshold. When the control is ‘on’, 
the length of the control vector is equal to the constraint limit while 
its ‘direction’ is governed by the ‘direction’ of the system adjoint 
solution. 

The usually difficult problem of obtaining the initial conditions for 
the adjoint solution is reduced to the problem of maximizing a par¬ 
ticular function. It is shown that this function possesses a unique 
maximum and that its gradient assumes an especially simple form. A 
direct synthesis procedure then follows by applying the method of 
steepest ascent. The method developed is particularly suited for digital 
computer solution. 

Sommaire 

Ce rapport presente un probleme de pilotage de missile en navigation, 
comme un probleme d’optimalisation. II s’agit de mettre le missile 
sur sa trajectoire nominale en utilisant un minimumde combustible. 
Une contrainte est appliquee du vecteur de la poussee. Prenant en 
consideration des perturbations par rapport a un regime nominal. 

La forme de l’optimalisation resulte du principe du maximum 
et constitue une fonction vectorielle de la solution adjointe du 
systeme. Cette optimalisation n’est realisee que lorsque une fonction 
scalaire de cette solution, depasse certain seuil. Quand cette con¬ 
dition est remplie, la longueur du vecteur caracteristique du systeme 
est egale a celle qui resulte de la contrainte, tandis que sa direc¬ 
tion est fixee par celle donnee par la solution adjointe du systeme. 
La difficult^ habituelle de determiner les conditions initiales de la 
solution adjointe du systeme se reduit au probleme de la maximali- 
sation d’une fonction particuliere. II est montrd que cette fonction 
ne possede qu’un seul maximum et que son gradient est d’une forme 
particulierement simple. Une procedure de synthese directe en decoule 
en appliquant la methode de la pente la plus raide. Cette methode 
se prete particulierement bien a un traitement au moyen d’un cal¬ 
culates numrique. 


Zusammenfassung 

Ein allgemeines Problem der Lenkung wahrend der mittleren Flug- 
phase (Startphase—mittlere Phase—Endphase) wird als optimales 
Regelproblem formuliert. Die Schwierigkeit besteht darin, den Flug- 
korper bei geringstem BrennstoffVerbrauch auf eine vorgeschriebene 
Flugbahn zu bringen. Der Schubvektor (oder der Vektor der Regelung) 
unterliegt einer Amplitudenbegrenzung. Die Linearisierung der Glei- 
chungen geschieht dadurch, dab man Storungen um die Nennwerte an- 
nimmt. 

Die Art der Optimalwertregelung folgt aus dem Maximumprinzip 
und ist eine Vektorfunktion der adjungierten Losung des Systems. Die 
optimale Regelung wird nur dann wirksam, wenn eine Skalarfunktion 
der adjungierten Losung des Systems einen festen Schwellwert iiber- 


schreitet. Bei wirksamer Regelung ist die Lange des Vektors der Rege¬ 
lung gleich der vorgegebenen Begrenzung, wahrend seine „Richtung“ 
von der „Richtung“ der adjungierten Losung des Systems abhangt. 

Die normalerweise schwierige Gewinnung der Anfangsbedingungen 
fur die adjungierte Losung wird auf die Maximierung einer bestimmten 
Funktion zuruckgefiihrt. Es zeigt sich, daB diese Funktion ein ein- 
deutiges Maximum besitzt und daB ihr Gradient eine besonders ein- 
fache Form annimmt. Daraus folgt unmittelbar ein Syntheseverfahren 
nach der Methode des steilsten Anstieges. Die hier entwickelte Methode 
ist besonders gut fiir die Losung auf Digitalrechnern geeignet. 


Introduction 

Recent advances in optimal control theory 1 " 3 have led naturally 
to an interest in applying these results to practical problems. 
The need for application studies is especially strong in the field 
of missile and space vehicle guidance and control, where such 
factors as fuel consumption, payload, mission time, and target 
errors are critical. Some results have been obtained in this 
field 4 " 7 . 

This paper presents a synthesis of the control function which 
is optimal in the sense of minimum fuel for the midcourse phase 
of a broad class of space missions. The central result of the paper 
is the development of an iterative computational procedure for 
synthesizing the optimal control. The procedure is simple in 
form and well suited for digital computation. 

Midcourse Guidance as an Optimal Control Problem 

A number of space missions may be subsumed under the 
configuration shown in Figure 1. The problem is that of trans¬ 
ferring a space vehicle from one moving point A to another 
moving point B. Both A and B may lie on bodies which are 
spinning as well as moving along their respective orbits. 

Typically, the mission and its associated guidance operations 
are separated into three phases: launch or boost, midcourse, and 
terminal. These are shown in the figure. If the launch phase 

Destination 



Figure 1. General configuration for space missions 
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could be executed perfectly, the vehicle would ‘free-fall’ to the 
destination point without requiring any corrections. However, 
imperfect launch guidance, resulting from sensor errors, in¬ 
correct thrusting, external disturbances, etc., causes errors to 
exist at the conclusion of this phase. Hence, midcourse correc¬ 
tions may be required. The term errors is used here to denote 
position and velocity deviations from the ideal ‘free-fall’ tra¬ 
jectory which is hereafter termed the nominal trajectory. 

There are basically two major problems associated with the 
overall guidance problem for each phase of a space mission. The 
first deals with the sensing and processing of information to 
determine the vehicle’s state, such as its position and velocity. 
The second is concerned with utilizing this information to guide 
and control the vehicle so that the mission objectives are achiev¬ 
ed. This paper presents a solution to the second problem for the 
midcourse phase, under the assumption that the position and 
velocity errors are known at the termination of the launch phase. 
These errors are assumed to be small enough to permit linear¬ 
ization of the vehicle’s equations of motion about the nominal 
trajectory. 

In order to allow freedom in the terminal phase for manoeu¬ 
vring and landing or docking, it is required that the midcourse 
corrections reduce the errors to zero at the end of the midcourse 
phase. In addition, it is required that the midcourse phase be 
executed in a fixed time equal to the time for the nominal flight. 
Physically, these two requirements mean that the vehicle will 
arrive at the desired destination point with the same velocity 
and at the same time as a vehicle following the nominal trajec¬ 
tory. Thus, the task of the terminal phase will be reduced con¬ 
siderably. 

It will be assumed that the midcourse phase is to be executed 
using a minimum amount of fuel in order to maximize the pay- 
load. In addition, it is assumed that the corrective thrust magni¬ 
tude is constrained for obvious practical reasons. 

The equations of motion of a space vehicle subject only to 
gravitational and propulsive forces may be written as 

Xi=ft<,x u x 2 ,x 3 ,t)+^Ti(f) i- 1,2,3 (1) 

where x^ x 2 , and x% are the coordinates of the centre of mass 
of the vehicle in a Cartesian inertial coordinate system and the 
dots denote differentiation with respect to time. The gravi¬ 
tational accelerations, as represented by the /*, are time de¬ 
pendent since the attracting bodies, e.g. planets, may be moving. 
The T t (t) represent the propulsive forces, and m is the mass of 
the vehicle. It is assumed that the mass of the fuel consumed 
during midcourse flight is small compared to the total vehicle 
mass. Thus, m is essentially constant during the mission. 

De “" 8 1-1,2,3 (2) 

the six equations, (1) and (2), may be written in vector form as 

x=/(x,0 + Tt(0 (3) 

Here x is the vector (x u ..., x 6 );/depends only on the first three 
components of x (as well as time); and the vector T(t) is related 
to the Ti(t) of (1) in an obvious way. 

It is sometimes more advantageous to use polar or spherical 
coordinates rather than Cartesian coordinates as in (1). In 
vector form an equation similar to (3) is obtained: 

y=P(y>t)+Q(y)T(t) (4) 


For the sake of generality, it will be assumed the equations of 
motion are of the form of (4). 

Let y = Y+ Sy where Y represents a nominal trajectory 

satisfying the free-fall equation 

y=p(y, 0 


Since y satisfies (4), Sy satisfies the equation 


Sy=(^Jdy+Q(y)T(t)+... (5) 

where (0 P/dy) is a 6 x 6 matrix of partial derivatives (evaluated 
along the nominal trajectory Y ), and the dots on the right denote 
higher order terms in dy. 

The term Q (y) T(t\ which represents a propulsive correction, 
will be considered to be small of the first order. Therefore, the 
difference [Q(Y)-Q(y)']T (t) 

will be considered to be a ‘second-order’ term. Thus, neglecting 
higher order terms, eqn (5) can be written as 


Defining 


(6) becomes 


5y = (^j5y + Q(Y)T(.t) 

x(t)—8y(t) -4 (f) == (q^j) 

u(t)=T(t) B(t) = Q(Y) 

x(t) — A (t) x(t) + B (t ) u (t) 


( 6 ) 


(7) 


As an example consider the two-dimensional, restricted two- 
body problem. Let r x and 6 1 be the polar coordinates of the 
centre of mass of the vehicle in an inertial coordinate system 
with origin in the centre of the attracting body. Let u t and u 2 be 
the components of the vehicle’s radial and tangential thrust, 
respectively (see Figure 2). Letting r x = r 2 and = 0%, it is well 
known that the equations of motion take the form 





r 2 

d 

r 2 


n2 V 

? T02-^ 

df 



0 2 
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where m is the vehicle’s mass, y, 
gravitational constant, and M is 
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GM where G is the universal 
2 mass of the attracting body. 
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Linearizing (8), with R, 0, R, and 0 the values of r l9 6 lt r 2 , 
and 0 2 along the nominal trajectory, respectively, yields 



X 1 


0 

1 


0 0 

d 

x 2 


A 2 , 2 F 

0 +jp 
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0 2 Rd 

dt 

x 3 
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+ 
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0 0 
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0 — 



mR_ 


u i 

u 2 


where x 2 = dr l9 x 2 = <5r 2 , x 3 = dO l9 and x 4 = Sd 2 . 

Returning to the general case, it will be assumed hereafter 
that x(t) is an ^-dimensional column vector, the state vector; 
u(t) is an r-dimensional column vector, the control vector; and 
A(t) and B(t) are nxn and n x r matrices, respectively. 

Since the time interval over which midcourse guidance is 
performed is fixed, it is assumed that 0 < / < t in (7), where r 
is a l^nown constant. The deviations which exist at the initiation 
of midcourse guidance are denoted by x(0). To null these errors, 
it is required that x(t) = 0. 

An amplitude constraint is placed on the thrust or control 
vector by requiring ||«(f)|| < A, where A represents the maxi¬ 
mum thrust available. The symbol || || denotes the Euclidean 
norm given by 

ii «(0 ii=V u i(o+ ••• +u 2 r (t) 


where the u { (t) are the components of u(t). Note that ||«(/)|| is 
simply the length or magnitude of the control (thrust) vector. 
Replacing the matrix B(t) by A B(t), and u(t) by (1/A) u(t), the 
constraint can be assumed to be normalized, and of the form 

II “(Oil < i. 

Assuming a constant rate of expulsion of fuel, the am ount 
of fuel consumed during the midcourse phase is proportional 
to the time integral of the length of the control (thrust) vector. 
Thus, the cost function to be minimized is given by 


S= 


HttCOII 


(9) 


The formal mathematical statement of the problem is now 
presented. Given the linear dynamical system of (7), it is desired 
to drive the system from a known initial state x(0) in phase 
space to the origin in a fixed time x such that the cost function (9) 
is minimized subject to the constraint ||h(/)|| < 1 for all t 

0 < t < T. 

Controls for which ||«(0|| < 1 and for which each compo¬ 
nent of u(t) is measurable for 0 < t < x are termed admissible. 
The set U of all such controls is defined by the relation 


U—[u(t): ||u(f)||<l, u (t)measurable, 0<f<r] 

where u(t) is an r-dimensional column vector. 

An admissible control which minimizes the integral in (9) 
and drives the system given by (7) from the initial state x(0) to 
the origin in time r is called an optimal control. 


Derivation of the Optimal Control 

The response of the system given by (7) is 
x(t) = X(t) 


x ( d ) + X 1 (s)B (s) u (s) d s 

0 


( 10 ) 


where X(t) is the n x n matrix solution of 

X(t)=A(t)X(i), X (0)=I 


where I is the identity matrix. 

Since X(t) is non-singular, x(r) = 0 if and only if 


— x( 0 ) = 



(s) B (s) u (s) ds 


( 11 ) 


In other words, for an admissible control u(t), (11) determines 
the initial state from which it is possible to reach the origin in 
time t using the control u(t). 

Let an additional state variable x,^ be defined by 


Si-t-l 


( 0 = 


||u(s)|| ds 


Clearly, x ra + x (T) is the cost function. 

Now consider the set Q(t) defined by the relation 

f3(r) = 


X \t)B(i)u{t)dt, ||u(t)|| dt 


: u (0 admissiblej 


A point (x 1; ..., x n , x n + x ) belongs to I2(r) if and only if there is an 
admissible control u(t) which drives the initial state (-x x ,..., 
— x n ) to the origin in time x with cost x, i+1 . 

Since - u(t) is admissible if u(t) is, and since ||-«(/)|| = 
II“(Oil, it is clear from the definition of Q(r) that this set is 
symmetric about the cost (x n+1 ) axis. 

It is shown in the Appendix that Q(r) is convex, closed, and 
bounded. A typical three-dimensional representation of !2 (t) 
is given in Figure 3 for a second-order system. For higher-order 
systems the simple two-dimensional representation of Figure 4 
will be helpful in the discussions which follow. 



Figure 3. Representation of Q (t) for second-order systems 



Figure 4. Simplified representation of Q (r) for higher-order systems 
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For some initial states it will be impossible to reach the 
origin even if full control \\u(t)\\ = 1 is utilized throughout the 
interval [0,r]. The set of all such initial states is termed de¬ 
generate. Only cases where the initial states are non-degenerate 
will be considered here. 

If a non-degenerate x (0) is given, the minimum cost to reach 
the origin in time r is the least x n ^ (r) for which (— x ( 0 ), 
x n+1 (r)) e Q (r). This cost is denoted by x° n -fi (r). Obviously, 
(— x ( 0 ), xVfi ( T )) is a boundary point of Q (r). 

Let y 0 = (— x (0), x 0 ^ (r)). Since Q (r) is convex, there 
exists at least one (n + l)-dimensional row vector ?]* such that 

r\*-y Q >r\*-a> ( 12 ) 

for all cog Q (r). That is, a hyperplane of support may be con¬ 
structed at y 0 with rj* a vector normal to this hyperplane at y 0 . 
This result has a simple geometric interpretation which is given 
in Figure 5. The hyperplane of support is ‘tangent’ to the bound¬ 
ary of Q (r) at y 0 . The vector rj* is normal to this plane and 
is directed ‘out of’ or ‘away from’ & (r). Observe that the 
components of this vector are proportional to the direction 
numbers of the line ‘normal’ to the boundary of Q (r) at y 0 . 

It is geometrically obvious (see Figure 5) and easily shown 
that the in + l)st component of r\* is non-positive. With little 



Figure 5. Geometric interpretation of hyperplane of support and normal 
vector rj* 


loss of generality, it may be assumed negative. Since the length 
of >/* is obviously immaterial, the (n + l)st component of 77 * 
is taken to be -1. Henceforth, it will be assumed that all 
‘normal’ vectors to Q (t) with negative (n + l)st component 
have this component equal to — 1 . 

The function (of of) '//* * co attains its maximum 8 in Q (r) 
w hen co = y 0 . In general, for coeQ (r) given by 


(JQ-- 


JXT" 1 CO J5 (t) M (0 dr. 


LJ 0 


II U (Oil df 


* co becomes 


rj -co — 





(13) 


where rj* — (t), — 1 ). ...... 

It is desired to select an admissible control u ( t ) which 
maximizes 7 j* * co . This function will be maximized by maximiz¬ 
ing the integrand of (13) for all t, 0 < t < r. 

Let 

f = (t)B(t)u (0-1|w (Oil O 4 ) 

and define 

(15) 


which is an r-dimensional row vector. 
Substituting (15) into (14) gives 


as the function to be maximized by the choice of an admissible 
control u (t) over the interval [ 0 , t]. 

If a (f) = 0,/ (t) will clearly be maximized by setting u (f) = 0. 

If a (0 # 0, / ( t) will be maximized only if u (t) has the same 
direction as a it) in which case 

(0 * u (0 = II(0II ■ II«(0II 

and 

/ (0HI« (Oil Clio (Oil -1] O 7 ) 

The maximization of/ (?) is now separated into the following 
three cases: 

(a) ||a (Oil >1 

(b) II a (Oil <1 
00 Ik ( 011=1 

Case ( a) 

Whenever ||a(*)||>l, fit) is maximized by setting 
|| u (t )|| = 1 , the maximum permissible. Hence, since the two 
vectors are in the same direction, the choice of u (t) is 

a'(t) 

M() Ik (Oil 

where the prime denotes the transpose. Note that the com¬ 
ponents of u ft) are simply the components of a it) normalized 
so that || u (OH = L 

Case ib) 

Whenever ||«(0||<L fO) is maximized by setting 
|| u (r)|| = 0 , or equivalently, 

w(t) = 0 

Since u it) is also zero for the special case where || a (Oil = 0, as 
seen above, then u it) = 0 whenever 0 < || a (Oil < 1« 

Case ic) 

Whenever || a (f)|| = 1, (17) reveals that the choice of 
|| u (OH is immaterial as long as u (t) is admissible. The system 
is defined to be regular if the set of points in [0, r] at which 
|| a (Y)|| = 1 is of measure zero for every vector rj*. Only regular 
systems will be considered here. For such systems, the control 
which maximizes r) * • oo if or cog Q (r)) is determined almost 
everywhere. Since sets of measure zero play no role in the 
systems under consideration, they will be ignored, so that the 
control which maximizes rj* * co, that is, the optimal control, is 
uniquely determined. For the sake of completeness, however, 
the choice u it) = 0 will be made whenever || a(f)|| = 1* 

Note that because of the uniqueness, inequality (12) can be 
replaced by the strict inequality 

rj*-y 0 >ri*-CD (18) 

for all co e 12 (t) different from y 0 . 

Recalling the definition of a it) from (15), and summarizing 
the above results, the admissible control which maximizes 
rj* * co in (13) is given by 

\JrrX 1 (f) B (QT if 11 , 7 . x - 1 (0£ (Oil >1 

u(t)= | llr*- 1 0)5(011 (19) 

I 0 if Ih-z -1 ( 0^(011 <1 


/ (0 = a (0 - M (0 — II M (Oil 


(16) for 0 < t < T. 
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Note from (19) that whether or not the control is ‘on’ is 
governed by whether or not || rj • X- 1 (t) B (r)|| exceeds a fixed 
threshold. Moreover, when the control is ‘on’, the system utilizes 
the full capability of this control by making || u{t)\\ — 1 and 
changing only the ‘direction’ of the control in accordance with 
the components of y * X~ x (/) B (t). 

Synthesis of the Optimal Control 

Since y is the only unknown in (19), the problem of syn¬ 
thesizing the optimal control is that of determining an y that 
corresponds to a given initial state x (0). 

Another way of looking at this problem is to observe that 
the transpose of the row vector y • X~ x ( t ) is the solution of the 
homogeneous system adjoint to (7): 

^ (0 = - A (0 ^ (0) = rj f 

where the prime denotes the transpose. Hence, the synthesis 
problem is equivalently that of determining the initial conditions 
on the system adjoint. 

Now consider an (n + l)-dimensional vector of the form 
2* = (2,-1) (20) 

where 2 = (2 ls ...,2J. 

For an arbitrary 2, let the function G (t, 2) be given by 

if ||2-X- 1 (t)B(t)||>l 

if ||2-X~ 1 (t) J B(t)||<l 

which is obtained from (19) by replacing r) by 2. For a regular 
system, G (Y, 2) is unique almost everywhere for every vector 2. 
Define the vector function 

z(t,2*)= X~\t)B{t)G(t,X)dt, P||G(U)||dfl (21) 
LJ o Jo J 

In the same way as relation (18) was derived, the relation 

2*-z(t, 2*)>2*-( (22) 

for all t £ Q (t), 'Q z (r, 2*) can also be obtained. Hence, 
z (r, 2*) is on the boundary of Q (r). The hyperplane of support 
to Q (r) at z(r, 2*) has the normal vector 2*. This hyperplane 
is denoted by B (2*) as shown in Figure 6. 


^n+1 



Figure 6. Representation of the problem of determining rj* 


G(t,X) — j ||2-Z -1 (t)F(t)|| 
0 


The geometric formulation of the problem of determining 
an 7]* associated with a specified initial state x(0) is depicted 
in Figure 6. If rj* is known, the corresponding point on the 
boundary of Q (r) has the coordinates 

z(t, >/*) = (-*(0),x° +1 (t)) (23) 

where x Q n ^ (t) has been defined previously. The hyperplane 
of support at this point is denoted by A. 

Let 2* be any vector as defined in (20) such that z (r, 2*) f 
z (r, ??*). As shown above, z (r, 2*) is on the boundary of Q (r). 
If £ = <A> • • •» Sn> f«+i) is any point in the space E n+1 which also 
lies in B (2*), then 

2*-z( t,A*)=A*-f (24) 

This is simply the equation of the plane B (2*). The line through 
the point (— x (0), 0) parallel to the (n + l)st coordinate axis 
intersects B (2*) at the point £ = (- x (0), | M+1 ) as shown in the 
figure. Substituting £ = (- x (0), f n4l ) into (24) gives 

2*- 2 (t,2*) = A*-(-x(0),£„ +1 ) (25) 

Since z (r, 2*) # z (r, rj*), it follows from (22) that 

2*-z(t,2*)>2*-zM*) (26) 

Substituting (23) and (25) into (26) gives 

2* • ( - x (0), {,+0 > 2* • ( - x (0), x ° n+1 (t)) (27) 

Expanding (27) yields 

-2-x(0) —£„ +1 >-2 -x(0)-x„° +1 (t) 
which simplifies to 

+ 1 < X n+ 1 ( T ) (28) 

This result is geometrically clear from Figure 6 and indicates 
that the value of g n+1 cannot exceed x° w +i (t). 

Solving (25) for £ n+1 gives 

£„ +1 = -2*-(x (0), 0) - 2* • z (t, 2*) (29) 

It is clear from (29) and Figure 6 that £„ +1 is a function of 2 
since 2* = (2, — 1). 

Now, if z (t, 2*) = z (t, if ), it readily follows that 

4+i(2) = x„° +1 (t) (30) 

However, (30) holds if and only if z (r, A*) = z (r, y*). 

Hence, the problem of synthesizing the optimal control has 
been reduced to the problem of finding an (n + l)-dimensional 
vector A* = (A, — 1) which maximizes g n+1 which is given in (29). 

Computation of the Optimal Control 

Since the (n + 1) st coordinate of A* is fixed, the maximization 
of £ n+1 in (29) is actually with respect to the ^-dimensional 
vector A. 

From (21) 

f x 

2*• z( t, 2*) = [2-X- 1 (0B(f)G(a)—||G(f,2)ll]dt (31) 

Jo 

in which 2 is the only variable. To simplify the notation, let 

s(2) = 2*-z(t,2*) (32) 
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Substituting (32) into (29) and expanding the result gives 

£„ +i a)=-A-x(0)-g(l) (33) 

The maximization of (A) may be performed readily by 
utilizing the method of steepest ascent. The variable A is made 
a function of some parameter a and the differential equation 

^=kV£„ +1 (A) (34) 


boundary point of Q (t). From (22) it is clear that z (t, A 0 *) is 
also a boundary point of Q (r). Hence, 

z(t, X°*) = z(r 9 ri*) 

and, therefore, G (/, A 0 ) is the optimal control. 

Substituting (35) into (34) yields 

o 


dA 

da 


= -fc 


x(0) + 


is solved for A (a). Here A is an /z-dimensional column vector, 
k is some positive constant, and V denotes the gradient. Then, 
under the conditions discussed below and a suitable initial 
choice of A, that is, A (0), it can be shown that the desired rj 
is given by 

r\ — lim A (a) 

<7-» 00 


In particular, it will be shown below that if this limit does exist, 
it is precisely the desired rj. Substitution of this rj into (19) then 
gives the optimal control for the problem. 

It can be shown 8 that Vf n+1 (A) is a continuous function of A 
and is given by 


V£ n+1 (A)=-x(0)~ 


(35) 

o 


Hence, from (34) dA/d a is also continuous. 

Assume 

limA(<r) = A 0 (36) 

a-* co 

where A° is a constant ^-dimensional vector. Since (A) is 

continuous, 

lim V£„+i (A(c)) = V<!;„ +1 (A 0 ) (37) 

<x—► oo 

From (36) and (37) it then follows that as X X°, dX/da -> 0, 
lim V(J n+1 (A(tr))=0 (38) 

<r-> oo 


Hence, for X = A 0 in (35), it is clear that 


-x(0) = 


0 


(39) 


This means that G (t, A 0 ) is precisely the control required to 
drive the system to the origin from x (0) in time x. 

Let A 0 * = (A 0 , - 1). Then, from the definition of z (r, A*), 
it follows that 


z(x,X°l = 


X _1 (0 B(t)G(t,X°)dt 


£iiG(a°)ii d tj 

(40) 


Substitution of (39) into (40) gives 
z(t, A 0 *)=f —x(0) 


, Jj|G(a°)iidt] 


From (23) (see also Figure 6), 

z(x,r,*) = (-x(0),x° n+1 (x)) 


Hence, the first n coordinates of z (t, A 0 *) are the same as the 
first n coordinates of z (r, rj*). By hypothesis, z (t, r/*) is a 


which may be rewritten as 
dA 


du 


= -fcX _1 (L>iX(0 


x(0) + 


Z' 1 (t)B(OG(i,A)dt 


(41) 

From (10) the term in brackets is the solution of (7) for x (t) 
where the choice of A determines the control. Since this x (r) 
depends on A which is a function of a in (41), define 


x (t, A) = X (t) 


x(0) + 


X- x (t)B(OG(f,A)dt 


Substituting (42) into (41) yields 
dA 


dcr 


= -kX~ 1 (x)x(x,X) 


(42) 


(43) 


The discrete form of (43) is 

A (l ‘ + 1) - A (0 = - o: ” 1 ( t) x (t, A (0 ) (44) 

where K is some positive constant and i is the index of iteration. 

A computational procedure is now clear. An initial ‘guess’ 
A (0) is made and substituted into (19) to determine the correspond¬ 
ing control. This control is then applied to the system of (7) to 
obtain x (r, A<°>). This result is substituted into (44) to compute 
AW, the first iteration on A. The cycle is then repeated using 
(44) to iterate on A each time. The process continues until 
|| ^(*+i) _ || is less than some specified constant. Then, the 

optimal control is obtained from (19) using the last iteration on A. 


Discussion of Results 

This paper has presented a synthesis procedure for a class 
of minimal effort controls. While the results are fairly general, 
it will prove instructive to discuss them in terms of the guidance 
application which motivated the study. 

By minimizing the fuel consumed in the midcourse phase, 
more space is made available on a given space vehicle for such 
items as life support equipment, communication sytems, and 
scientific instruments. The scope and efficiency of space mis¬ 
sions are, of course, directly dependent on the availability of 
additional equipment. While it is desirable that the space saved 
by using the optimal system be sizable, a saving of only a few 
pounds may permit the inclusion of experiments not possible 
otherwise. An important factor in space and weight considera¬ 
tions is whether or not the computations required can be done 
by equipment already on board the vehicle. If additional com¬ 
puter equipment is required, the question of savings in weight 
and space must be examined carefully. 

Just how close the terminal error x (t) will come to being 
zero in any practical system depends on a number of factors. 
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Some of the more important factors include the accuracy with 
which the initial error x (0) can be measured, the accuracy with 
which the control can be computed and the capability of the 
midcourse propulsion system to implement this control. 

The first of these factors depends upon the performance 
limitations of measurement schemes and the associated sensors. 
In general, the longer one is willing to wait, the better will be 
the estimates of position and velocity errors. However, in 
waiting, initial errors are permitted to continue uncontrolled. 
The trade-off is obvious. One possible approach is to divide the 
mission interval [0, r] into two or more sub-intervals. Then, 
while control is being executed over one sub-interval on the basis 
of earlier error estimates, data can be sensed and processed to 
improve the error estimates for succeeding sub-intervals. While 
the system is now optimal over each sub-interval rather than over 
the entire interval [0, t], the control is based on more accurate 
position and velocity data. Since the control operates ‘open-loop’ 
once X is determined, this latter approach also permits the system 
to control for disturbances occurring in previous sub-intervals. 

The accuracy with which the control can be computed is 
governed by how rapidly the solution of (43) converges. In addi¬ 
tion, the amount of computing time allowed before control must 
be initiated as well as computer speed are key factors. Studies 
are needed to determine how small || — X to || must be 

before computation is terminated. This information is essential 
to ensure that ||* (r)|| can be brought within reasonable limits. 

As a result of the ‘on-off’ nature of the optimal control, 
non-throttlable propulsion can be used. Hence, the problems 
associated with controlling throttlable engines are completely 
circumvented. However, since the direction of the control is 
time-varying, some means must be provided for controlling 
this direction. 


Appendix 

In this appendix it will be proved that the set Q (r) is convex, 
closed, and bounded. Since the components of X~ l (t), B (t), 
and u (t) (so long as u is admissible) for 0 < t < r, are uniformly 
bounded, it immediately follows that Q (r) is bounded. Let 

L(u(t))=\ X-'WBitMQdt 

A J ° 

and 

■ F (w(0)=J o II«(Oild t 

To show that Q (t) is convex it must be shown that if y 1 
and y 2 are any two points in Q (r), then y* = ccy 1 + also 
belongs to Q (r) whenever a > 0, /? > 0, oc + P = l Since 
/e-Q(r), y* = [L («< (/)), F(u*m, where F (t) is admissible 
(i = 1, 2). Let u*(t) = ocu\t) + p u 2 (t)- then 

ll M *(0ll = l|aw 1 (0+/?M 2 (0|| <a [|w 1 (?)II + j® ||w 2 (01| <a + >5 = 1 

for all t (since u 1 and u 2 are admissible), so that u*(t) is admissible. 
The above inequalities also imply that F(u*)<, oc F(u l ) + ft F(u 2 ). 
Since L is a linear operator, L («*) = ocL (u l ) + (3L (« 2 ). Thus 
y* = [mL (k 1 ) + 0L (u 2 ), oc F (a 1 ) + [SF{u 2 )] = [L («*), y* +1 ] 
where y* n+1 > F(u*). 

An admissible control u** (f) will now be constructed such 
that L(u*) = L (u**) and F («**) = y* n+1 , i. e . such that 


y * = \L(u*% F («**)], so that y*e Q (x). Since ||« !: (r)|| < \ 
for all t, it follows that F(F) < t (i — 1, 2). Thus 


y* + 1 =ocF(u 1 ) + [IF (u 2 ) <(a+jj)z = x 

To construct u**, a control v (t ) will first be constructed, with 
the property that \\v (Oil = 1 for all t (so that F(v) = r), and such 
that L(v) = L (w*). Let 

[w i!t (0 if i/*(0A0 
l(l,0,...,0) if u*(t)=0 
so that 1 > || ii (0|| > 0 for all t. It is clear that u (t) is admissible 


u (0 = 


and that 


Let 


u *(t)=\\u*\\ 


u (0 

Mol 


7(0 = X- 1 (0B(0t|: (0 


II«(Oil 

Clearly, rj (t) is measurable and bounded, and, by (45), 


(45) 

(46) 


L(u*)=j^(t)\\u*(t)\\dt 

According to Lemma 2 of ref. 9, there is a measurable function 
oc (/) with | « (0 I = 1 such that L («*) = \ T oV (t)oc (/) d/. Let 

u{t) 


= 1 for all t, and, by (46) and (47), 


(47) 


Then ||» (OH = | « (/) | 

L (u*y = Jo (0a(0 df= J X~ 1 (t)B (; t ) v(t)dt=L(v) 


and v ( t ) has the desired properties. Repeating the above con¬ 
struction on sub-intervals of [0, r], it is possible to find admis¬ 
sible controls v s (t ), for 0 < t < s (s < r), such that [|C (?)[[ = 1 
for all t < s, and 


X 1 (t)B (t) u* (t) dt = 


Let 


X- l (i)B(t)v s (t)dt 


(u s (0 for 0 <t<s 
In* (0 for s<t<t 


Then it is easily seen that L (u s ) = L («*). Define 0 (s) = F(u s ) 
so that 6 (0) = F(u°) = F(u*) < y \ +1 and 0 (r) = F(F) = 
r — y*n+i- It is obvious that 9 (s) is a continuous function of s, 
so that 6 (s 0 ) = y * n+1 for some s 0 , 0 < s 0 < x. If u**(t) = 
u s « (/), u** has the promised properties. 

To prove that Q (r) is closed, let / = [L (id), F(u% j = 1, 
2,..., u 1 ( t ) admissible, be a sequence of points in Q (r) with 
y ’ r^J*' 11 must be shown that y*eQ( r). Since | w 3 '.,- (t) | < 

II^(Oil ^ 1 for all t,j, and i — 1, ... r, the functions u[ ( t ) are 
uniformly bounded in the norm of the Hilbert space U (0, r). 
By a well-known property of L, 2 , there exists a subsequence 
u’ ( t) such that uf ( t ) -»■ u ( * ( t ) weakly for every i and some 

k->oo 

functions u* (t) inL 2 . It will simply be assumed that z4(0“*#<*(0 
weakly for each i. By definition of weak convergence, L(u } ) -+ 
L(u'), where u* — « r *). It is is a further consequence 
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of the weak convergence that ||«*(f)|| < 1 for almost all t , 
and that F (u *) < ^ F (u 1 ). Thus, y * = [lim L (??'), lim F (if )]= 
[L («*), y*„+i] with y* n+1 > F («*). Now an admissible control 
«**(/) such that y* = [L (u**), F(it**)] can be constructed just 
as above, thereby proving that y* eQ (t) and that Q (r) is closed. 
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DISCUSSION 


M. Athans, M.I.T. Lincoln Laboratory , Lexington , Massachusetts , 
U.S.A. 

In the literature on optimal control, the paper by Meditch and Neu¬ 
stadt is one of the few concerned with control constraints of the type 

|K0||<L .... , lt 

The purpose of this discussion is to present some recent results 
which provide the solution to a class of problems using the same 
control constraint. 

Consider the system 

x(t)=f[x(t);t]+u(t) 

where x ( t ), X (/), u (?), and / are n dimensional vectors and t is the 
(scalar) time. 

Assumption 1 The control vector u ( t ) has n non-zero components 
ii x (/),.. u n (?) and is constrained by 

||u (t )|| = V^(f)+ + w„ 2 (0< M for all t 


Assumption 2 

<f[x(t);tlx(t)>=gl\\x(t)\\;t'] 

for all x it) and t, where < /, x > is the scalar product of the vectors 
/and x, || x (f) || is the Euclidean norm of the vector x (/) and g is the 
scalar function of || x (t) || and t. 

Under these assumptions it has been proved 1 that the control 


u(t) = —M 


x(t) 

II x (Oil 


will force any initial controllable state to the origin, 0, in minimum 
time. 


In the special case 

g[l|x(OII;d=° 

for all x ( t) and t, the time-optimal control u (t) = — Mx (t)l\\ x (?) || 
is also fuel-optimal to the origin in the sense that it minimizes the 
functional 

IN (Oil it 

J o 

This fuel-optimality has also been proved 2 * 3 . 

The theoretical results given above can be used for the optimal 
angular velocity control of a tumbling body in space, using gimballed 
reaction jets. 
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J. S. Meditch, in reply 

Dr. Athans’ interesting discussion is especially important since it 
presents one of the few classes of problems in which the minimal fuel 
control law can be obtained in closed form as a function of the ‘in¬ 
stantaneous’ state of the dynamical system, i.e., as a feedback law. 
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On Necessary and Sufficient Conditions for Time Optimal 
Control of Second-order Non-linear Systems 

E. B. LEE and L. MARKUS 

Summary 


Problems of constructing optimal control for non-linear second-order 
systems are considered. It is shown that, under appropriate hypotheses, 
extremal control for steering from a given initial point to a target is 
unique. This result is then used with previously known results and 
geometric methods to establish the global uniqueness of extremal 
control for certain non-linear second-order systems. It is shown how 
the optimum feedback control function can be constructed for this 
class of systems. Questions of existence and structural properties of 
optimal control are treated. 

Sommaire 

Les problemes de la realisation d’un reglage optimal pour les 
systemes non-lineaires du deuxieme ordre sont examines. Les resultats 
prouvent qu a partir d’une hypothese convenable, le reglage extreme 
pour guidage a partir d’un point donne jusqu’au but, est unique. Ce 
resultat est employe, avec des r&ultats anterieurement connus et avec 
des methodes geometriques, pour dtablir l’unicite globale du reglage 
extremal pour certains systemes non-lineaires du deuxieme ordre. La 
realisation de la fonction optimale de reaction pour des systemes de 
cette classe est ensuite developpee. Sont aussi discutees les questions 
d’existence et de proprietes de structure du reglage optimal. 

Zusammenfassung 

Der Aufsatz behandelt die Probleme der optimalen Regelung von 
nichtlinearen Systemen zweiter Ordnung. Es zeigt sich, daB unter ge- 
eigneten Annahmen ein eindeutiger optimaler Regelvorgang existiert, 
um von einem gegebenen Ausgangspunkt zu einem Ziel zu gelangen! 
Dieses Ergebnis wird zusammen mit bereits bekannten Resultaten 
und geometrischen Methoden dazu benutzt, Extremwertregelungen 
bestimmter nichtlinearer Systeme 2. Ordnung mit umfassender Ein- 
deutigkeit zu erstellen. 

Die Arbeit zeigt, wie der optimale Regelkreis fur diese Klasse von 
Systemen aussehen muB. Fragen der Existenz und strukturelle Eigen- 
schaften der optimalen Regelung werden behandelt. 


Introduction 

There are many physical systems whose evolution can be sum¬ 
marized as a solution of an ordinary second order differential 
equation. In certain of these systems there are parameters, due 
to design or otherwise, which can be selected to change the 
evolution of the system or analogously to change a solution of 
the differential equation. These control parameters may be 
subject to certain restrictions, for example, in the case to be con¬ 
sidered they are bounded in amplitude and assumed measur¬ 
able. Interest here is in constructing a controller (in terms of 
the differential equation, a function of two variables) which 
determines the control parameter so that each solution of the 
second-order differential equation passes to the origin (rest 
position) in finite time, in fact, the shortest possible time. 


In the next section the problem is formulated and the 
necessary conditions for time optimal control indicated. The 
optimal controls are shown to be ‘bang-bang’ with switches in 
magnitude occurring in a cyclic order with the zeros of the velocity 
coordinate when viewed as a function of time. There are certain 
questions concerning the existence of optimal control. Two 
important cases of the question of existence are resolved. 

The construction of a switching locus in the phase plane is 
considered as a possible way of constructing the optimal con¬ 
troller. It is shown that the switching locus contains a piecewise 
smooth curve which separates the phase plane. Cases in which 
the construction of the switching locus based on the necessary 
condition is also sufficient for optimal control are treated. An 
example is also shown. 

These results are essentially those which have been presented 
in detail 6 * 7 . The purpose here is to summarize the results for 
this problem and to explain their use and physical significance. 
Most of the proofs can be found in Lee and Markus 6 ’ 8 . 

Problem Statement and the Necessary Conditions 

Consider a dynamical system as described by the real second 
order differential equation 

*+/(*, x) — u 

or the equivalent system 

x = y 

<- S ) J V=~f(x,y) + u 

Here u is a scalar control variable, which is subject to the restric¬ 
tion — 1 S 1, and the real function f(x,y) is assumed 
differentiable everywhere with respect to the two variables x 
and y. The (x, y) plane is denoted here by J? 2 , the two-dimensional 
real number space. 

It is desired to synthesize 0 ( x , x) in the differential equation 
x+f(x,x) = 0(x,x) 

so that each of its responses (x (t), x (t)) = (x (/), y (t)) -+ (0, 0) 
in the minimum time. Generally the authors find controls 
u = u (J) for the system (S) so that the response (x ( t), y (?)) -+ 
(0, 0) in minimum time under the influence of u (t) and then 
indicate how u (t) can be used to construct the feedback controller 
0 (x, y). 

Consider the dynamical system (5). For a given initial 
condition (x 0 , y 0 ) at time t — 0, define A as the class of all 
(measurable) controls u (t) on various time intervals 0 g t g t lt 
with - 1 g u (t) g 1, for each 0 ^ t <> t x such that the response 
x W ~ ( x ( 0 , y (0) is defined on 0 g t ^ with x (0) = x 0 , 
y (^) I'd. and first reaches the origin at t ~ r,. The response 
(* (0, y (t)) is an absolutely continuous solution of (.S') with 
x (0) = x 0 , y (0) = y 0 . 
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A control u (?) on 0 g ? g ? x in A is called (minimal) time 
optimal in case, for each u (?) in A it is found that ?i = ?• 

By the Maximum Principle 1 an optimal control is necessarily 
an extremal control. That is: 

(?) There exists a nowhere zero, absolutely continuous, 
response rj (?) = (Vi (A % M) on 0 ^ ? ^ ?i such that x (?), 
rj (?), u (?) satisfy the equation system 


dH f . 

6 H 

5 — ’h — 

0)7 1 

0X 

0 H 

dH 

5 — */2 = 

0»72 l 

s y 


and, 

(??) H (rj (?), 5c (?), u (?)) = M(i 7 (?), x (?)) for almost all ? on 
0 ^ ? <; ?i. Here H (rj, 5c, u) = %x + %>’ = + % [-/(Xt) 

+ u] and M («, 5c) = max H ( 77 , x, «). 

-ISuSI 

Moreover, 

(in) M (rj (0), x (0)) — M(rj (?), x (?)) S: 0 for all ? on 
0 ^ ? £ ?,, and 7 ) (?), x (?) satisfying (i) and (?'?'). 

The authors state that a control u (?) on 0 ^ g ?j in d is 
a relay control in case there exists a finite number of times 
0 = t 0 ^ r! g ... t,. = ?!, such that on each open internal 

Ti-i < t < T it i = 1. k, 11 (?) equals either +1 or — 1 and 

switches value on successive intervals. 

From the Maximum Principle the following result is obtained. 

Corollary —Let u (?) on 0 ^ ^ ?i in d be an extremal 

control for the system (S). Then ?<(?) is equal to the relay 
control sgn {rj 2 (?)} almost everywhere on 0 ft 1 = h -(Hereafter 
it is to be assumed that the extremal control u (?) has been modi¬ 
fied on a null set so as to be equal everywhere to the corre¬ 
sponding relay control; that is, u(t ) = sgn [t] 2 (?)}• Here sgn 
[ ] = 1 if [] > 0, = - 1 if [] < 0, = 0 if [] = 0.) Also the 
response r / 2 (?) on 0 ft ? ft h has a finite number of zeros each 
of which is simple. 

From the Maximum Principle and certain results from the 
Sturm oscillation theory, the switching of the relay control can 
also be related to the geometry of the (x, y) plane. This result, 
which states that the zeros of y (?) and 77 2 (0 are interlaced, is 
basic in establishing the properties of the switching locus as 
dealt with later. 

Theorem 7—Let u (?) on 0 ft ? ft h in d be an extremal 
control for the system (S), and let x (?) = (x (?), y (?)) and 

(t) = (, ?1 (?), rj 2 (?)) be the corresponding extremal responses. 
Let ft, ft be times such that 0 ft ft < ft = h- Then 

(?) if r/ 2 (ft) = r /2 (ft) = 0 and if y (ft) = 0, then y (ft) = 0. 

(??) If r / 2 (ft) = ?/ 2 (ft) = 0 and if y (ft) # 0, then y (ft) # 0 
but there is a zero of y (?) on the open interval ft < ? < ft. 

(???) If y (ft) = y (ft) = 0, y (?) ± 0 on ft < ? < ft and if 
r/ 2 (ft) = 0, then r) 2 (ft) = 0. 

(?») If y (ft) = y (ft) = 0 , y (?) # 0 on ft < ? < ft and if 
772 (fj) 7 ^ 0 , then rj 2 (£ 2 ) # 0 , but there is a zero of (0 on 
open interval < t < £ 2 . 

Thus if the zeros of 7 (0 are isolated, they either coincide 
with the zeros of rj 2 ( t ) or else no zero of y ( t ) is a zero of (t) 
but these two sets of zeros are interlaced. 

Corollary —Let «(0 onO^^! in J be an extremal 
control for the system (S). Let the corresponding response be 


X (t) = 0 (0, y (0) and adjoint response be rj (t) = Ofr (0» ^2 (0) 
on 0 ^ t ^ h- Let ^2 (f) = 0 at * = £ 0 = f = *i* 

If y©> 0 , then jj 2 (f)<0. 

If y(O<0, then t) 2 ©>0. 

Remark— Already the optimal controls have been limited 
to being relay controls which switch magnitude once and only 
once in the upper half (lower half) plane along any optimal 
response which enters and eventually leaves the upper half 
(lower half) plane. The switches, if they occur, are from + 1 
to — 1 in the upper half plane and from — 1 to + 1 in the lower 
half plane for t increasing. Thus the optimal responses are 
composed of segments of the solution curves of 

(S±) y = -/(x,y)±i 

with switches determined by the switching locus W which is 
considered later. Hereafter £+ refers to the system (S) with 
u == + 1 anc i s - when u == - 1 . 

Domain of Controllability and the Existence of Optimal Control 

Consider the above system (5). The set # of all points 
(x 0 , y 0 ) in jR 2 for which there exists a measurable control 
-\^u (0 ^ 1 on a finite interval, 0 £ t £ t l9 steering (x (t\ 
y (?)) from (x 0 , y 0 ) to the origin is called the domain of (null) 
controllability. Here ‘steering’ means that there exists a control 
u (t) on 0 ^ t <; t v which, when substituted into the equation 
system (,S ) leads to a response (x (t ), y (0) tiiat starts at “ 
x 0 , y( 0 ) = y 0 and passes through the point ( 0 , 0 ) at t = t v 
Theorem 2— For the system (5) as above with /(0, 0) = 0, 
anc l — 1 g u S L the domain of controllability *€ is an open 
connected subset of R 2 containing the origin. 

Theorem 3 —Consider the system (5) with -1^«(0^1 
as above. Assume that /(x, y) is an attractive force with non- 
negative friction, that is 

xf (x, 0)>0 for x^0 

and 

|f-(x,y)>0 in R 2 . 

Then the domain of controllability = R 2 and, moreover, 
each point of R 2 can be steered to the origin by an optimal 
control u(t) in A. 

For the study of systems with a repulsive force the following 
conditions are assumed, which could be relaxed slightly in the 
manner suggested by Theorem 3. 

{So) y=-f(x,y)JM = QJ(x,y) 

differentiable and b//bx (x, 0 ) < — £ < 0 , b//by ^ 0 for some 
e > 0 . 

This system has the origin as the unique critical point. Near 
the origin the solution curve family of (*S 0 ) is topologically 
equivalent 8 to the solution curve family of the linear system 

(L) x = y 
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Thus there are four solutions of (S 0 ) which approach the origin 
as t -» oo or t— co. Call these curves I, II, III, and IV, 
depending on the quadrant in which they lie. A study of the 
geometry of (S 0 ), cf. reference 8, shows that I and III are defined 
and single valued over a half x axis. Also II and IV are single 
valued over an interval of the x axis and on these \y\ -> oo as 
|x[ increases. 

As in their previous work 7 * 8 , the authors find that the critical 
point and the curves I, II, III, IV are the only separatrices of 
(So) an d the remaining solution curves of (S 0 ) consist of four 
parallel canonical regions in the sense of Markus 9 . Thus (S 0 ) is 
globally homeomorphic to (Z) in R 2 . II and IV are called 
the principal separatrices of (*S 0 ). 

Theorem 4 —Consider the system 
(S') x = y 

y=~f (x, y) + uj (0,0) = 0,/ (x, y) 

differentiable and — 1 ^ u ^ 1. Assume / (x, y) is a repulsive 
force with non-negative friction, that is, 

(x,0)<-8< 0,~ (x, y) ^0 in R 2 for some e > 0 

Then each of (*S+) and (£_) is globally homeomorphic with the 
linear system (L) and has corresponding separatrices I+, II + , 
III+, IV+ and I~, II_, III—, IV_, as described above. The domain 
of controllability c € is precisely the open topological band 
bounded by the principal separatrices II+, IV+, and II_, IV- 
and the two critical points of 5+ and 5L. Furthermore, each 
point of can be steered to the origin by an optimal control 
u (t) in A. 


system S~ is started at / = 0). The first zero of ?j 2 ( t ) in / C 0 
is obtained and this determines the time interval through which 
one follows the solution x (/) of S~ which initiates on W+ 1 . 
Denote the endpoints of such solutions of S- as the reflection 
W~ 2 of W+ 1 (see Figure 1). 

Now reflect W- 1 through a conjugate interval along solutions 
of S+. That is, from each point on WJ- follow backwards in 
time along the corresponding solution of S+ for a time duration 
equal to the interval between zeros of rj 2 (t). This means that 

ni=rh^(x(f),y(t)) 

*l2=-ih + ri2^(x(t),y (?)) 

is used with x (?) = (x (/), y (?)) the appropriate solution of S+ 
initiating on IV- 1 and (0) = 1, rj 2 (0) = 0, to obtain the first 
zero of ij 2 (?) in ? < 0. Denote the reflection of IV- 1 alone S+ 
by W+\ 

If W+\ fV + 2 ,..., W+ k and fV\ W s ,..., W k have been defined, 
let W+ k+1 be the reflection of WJ‘ by the solutions of S+ and 
let W- h+1 be the reflection of W+ k by solutions of S-, through 
conjugate time intervals as indicated above. Of course it might 
happen that W+ k u WJ‘ is empty for every k greater than 
some positive integer. 

Theorem 5—Consider the system (C 1 or C 2 ... indicating the 
degree of differentiability) 


y=-f {X, y) + u,f (0,0)=0,/ (x, y) in C 1 


Construction and Properties of the Switching Locus 

Consider the set of all extremal relay controls in A, for the 
system ( S), steering points of the domain of controllability to 
the origin as above. The switching locus W is the set of all 
points in at which the corresponding responses x (t) fail to 
have derivatives, that is, W consists of the points at which the 
extremal responses switch from the solution famil y 0 f S+ to 
S~, or vice versa. 

For definiteness the origin in IF is included. A construction 
for W is now described. Let Wy 1 be the solution (or segment of 
a solution) of S+ through the origin and lying in the fourth 
quadrant x ^0,^0. Let W- 1 be the solution (or segment 
of a solution) of S- through the origin and lying in the second 
quadrant 

Now reflect W+ 1 through a conjugate interval along solutions 
of S —. That is, from each point on W+ 1 follow backwards in 
time along the corresponding solution of S- for a time duration 
equal to the interval between zeros of ?y 2 (t). This means that 

ri2=-rh + ri 2 -^(x(t),y(t)) 

is used with x (t) = (x ( t ), y (?)) the appropriate solution of S- 
and rj-y (0) — ~ 1, % (0) = 0 (note that the adjoint linear system 
is homogeneous and that the response x (t) to the autonomous 


with measurable control u(t) in - 1 g u ^ 1. The switching 
locus W is precisely the union of the sets W+ k u W- h for 
x = 1, 2, 3, ..., as described above. 

Proof— Using Theorem 1 it is noted that for each point P 
on W+ 1 in y < 0 can be selected an adjoint response y (t) = 
(Vi ( t ), ih (.t)) with % (t) vanishing when x (?) = P and nowhere 
else on fV+ 1 jn y < 0. Also if P is an endpoint of W+ 1 where 
y = 0, then y (t) can be selected so that y 2 (t) vanishes at times 
corresponding to x (?) = P and x (?) = 0. Then each point of 
W+ , including the endpoints, can occur as a switching point 
for a response to an extremal relay controller in A. Thus W+ 1 c= IV 
and similarly WJc W. 

Since every extremal relay control which steers a point of 
to the origin must have a response which enters the origin along 
either tV+ 1 or W-\ and since the response must follow alternately 
the solution curve families of 5+ and S-, or vice versa, with 
switches occurring at the zeros of % (?), it is concluded that W is 
the union of all W+ k and WJ C for k = 1, 2, 3,... (see Figure /). 

The smoothness properties of the switching locus will now 
be discussed. First consider the system 


y~ / ( x > y) + u ,f (0,0)=0,/ (x, y) in C 1 
with measurable controllers u (?) on — 1 g u ^ 1, and 

lr ( X’0)> e >0’fr(x,T)^o 
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for some a > 0. These hypotheses, somewhat stronger than 
those of Theorem 3, assure that each of the extremal systems 
S+ and S- has exactly one critical point, and these hypotheses 
generally simplify the exposition of the problem. For example, 
every solution curve of S+, other than the unique critical point, 
is either a periodic solution, or spirals inwards towards a limit 
cycle or approaches the critical points as t —► + o° • If 5 further, 
bf/by > 0, then the area condition of Bendixson states that 
there are no limit cycles. 


Uniqueness of Extremal Control (The Sufficient Conditions) 

Corollary to Theorem 6 —Consider the system 5 of Theorem 6. 
Assume that W= W (x) is a single-valued curve on - oo < x < oo. 
Then for each point P in R 2 there is one and only one extremal 
relay control in A which steers P to the origin. 

Using this corollary it is easy to verify the following theorem. 

Theorem 8 —Consider the system 


Theorem 6 —Consider the system 


^ ' y= —/ (x, y) + u 9 f (0,0)=0,/ (x, y)inC 2 

with bf/bx (x, 0) > a > 0, for some a > 0, and bf/by ^ 0 in 
jR 2 . Consider extremal controllers u(t) in A steering points of 
& = R 2 to the origin and construct the switching locus W as 
the union of the sets W+ k u W- k , as described in Theorem 5. 
Then W contains a homeomorph of a line, a piecewise C 1 curve 
which separates the plane. Also W lies in y ^ 0 for x ^ 0 and 
in y ^ 0 for x ^ 0. • 

Secondly, consider the control problem for the repulsive 
force case 

x — y 

(S) j> = -/ (X, y) + u,f (0,0) = 0,/ (x, y) in C 1 

and u(t) measurable on - 1 ^ u <> 1, as above. Assume 
bf/bx (x, y) < — a < 0, bf/by (x,y)> 0 in R\ for some 
s > 0. These hypotheses, somewhat stronger than those of 
Theorem 4, assure the existence of a band 83 between the 
principal separatrices of 5+ and S- , which is the domain *€ of 
controllability. For each point Pa83 there exists an optimal 
control u (t) in A which steers P to the origin. 

The switching locus W of S is defined as above. To construct 
W let W+ 1 be the solution of S+ through the origin and lying 
in the fourth quadrant. By a comparison of the slopes of the 
solutions of S+ and S-, it is found that JV+ 1 lies always interior 
to the band 83 in y < 0 (except for the origin) and y -> oo as x 
increases on W+ 1 . Let W — 1 be the solution of S~~ through the 
origin and lying in the second quadrant. Clearly W- 1 lies 
interior to 83 and y—> + oo as |— x| increases on W— 1 . The 
reflections W+ k are defined as above but it can easily be shown 
that these are all empty for k = 2, 3, 4,... and that W — W+ 1 
u W-\ 

Theorem 7—Consider the system 


^ 3 )=-/ (pc, y) + uj (0,0) = 0 J (x, y) inC 1 

with bf/bx (x, y) < — a < 0, bf/by (x, y) £ 0 in R\ for some 
s > 0. Consider extremal relay controls u (t) on — 1 ^ u ^ 1 
in A and construct the switching locus W. Then W — W (x) is 
a continuous single-valued curve over a segment of the x axis, 
and W separates 83 in two regions. Moreover 

wf ,JwUx) for x^O 
W |pf^(x) for xgO 

so W (x) in C 1 except at x = 0. 


W y = —/ (x, y) + u,f (0,0) = 0 J (x, y) in C 1 

with d f/bx (x, 0) > s > 0, for some s > 0, and bf/by ^ 0 in R 2 . 
Assume the switching locus W = W (x) is single valued on 
- oo < x < oo. Then for each point P in R 2 there is exactly 
one optimal control u (f) on - 1 ^ u g 1 in A which steers P 
to the origin. (The same conclusion holds if the switching locus 
W is single valued with respect to both flows S+ and S-, except 
for the initial curves W+ 1 , and W— 1 .') Define the synthesizer 

[1 for y < W (x) 

^(^y)={ 0 for y = W(x) 

l — l for y>W(x) 

Then the optimal response for P is the unique solution of 
x+f (x,x) = <P (x,x) 
initiating at P at t = 0. 

Special Hypothesis —Here it has been assumed that the 
switching locus W = W (x) is a single-valued curve for - oo < 
x < oo. This certainly holds if IV+ 1 and W- 1 extend to infinity 
without re-intersecting the x axis, and the adjoint equations are 
disconjugate, as in the next theorem, or in special cases where 
the damping is linear. The hypothesis also holds in the case of 
isochronous periodic solutions for S+, for example / (x, y) = x. 
For here consider the variational equations along S+ or S~; 


v 1 =v 2 


S/..1 0/., 2 


where v (f) = (r 1 (t), v 2 (e)) represents a vector and its parallel 
transport along the flow of S+. But 


and so 


v 1 rj 1 + v 2 rj 2 =constant 


for successive switching times t = 0 and t == t x . Now 


and so 


t; 1 (0)*y 1 (^ 1 )^0. 


Therefore, taking v (0) as the tangent vector along W\, v (f) is 
the tangent vector along W and thus W 1 ^ 1 is single-valued 
over the x-axis and JV± is single-valued over the x-axis. By in¬ 
duction we can then show that W = W (x) is single-valued on 

-OO < X < OO. 
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Corollary to Theorem 7—Consider the system S of Theorem 7. 
Then for each point P in there is one and only one extremal 
relay control in zl which steers P to the origin. 


If the solutions (f) of the adjoint equations can have at 
most one zero, then the switching locus W = W (x) is exactly 
W+ 1 u W- 1 . Consider the adjoint system 


Theorem 9 —Consider the system 

(S ) ±== y 

y=~f (x, y ) + u,f (0,0) = 0J (x, y)inC l 

with df/dx (x, y) < — e < 0, for some e > 0, and bf/by ( x,y ) g: 0 
in R 2 . Consider the domain = £3 of controllability for 
measurable controllers u (t) on — 1 ^ u g 1 in A, and construct 
the switching locus W — W(x). 

Then for each point P in & there exists exactly one optimal 
control u (t) in A which steers P to the origin. Define the syn¬ 
thesizer 

[ I for y<W(x) 

<P(x,y)=l 0 for y=W(x) 

l-l for y>W(x) 

Then the optimal response for P is the unique solution of 

x+f(x 9 x) = <P(x, x) 
initiating at P at t = 0. 

Remarks— The switching locus is single valued in the case 
in which the damping enters linearly and is sufficiently large. 
Consider therefore 


1 it=n 2 g'(x(t )) 

rh=-ni + br\ 2 

where x (t) is some solution of S+ or S~. Then 
n 2 -bfl 2 + g’(x(t))r\ 2 = 0 

But the solutions (0 have the same zeros as the solutions 

Z = Q~ bt l 2 T) 2 Of 

z + (g’(x(t))-^z =0 

Thus no (non-trivial) solution rj 2 (t ) has two zeros. Hence 
W = W (x) is exactly W+ u Wl 


Example—x + 2 x + arc tan x = u (t ), — 1 u (t) g l 
where 

71 j 71 

_ <arc tanx<-~ 

2 2 

Here 

g (x) = arc tan x,g (0) = 0 , g' (x) = ~ ™ g 1 


where b > 0 is constant and g (x) in C 1 with g (0) = 0, g' (x) > 0 
and | g (x)\ > 1 for large |x|. 

The domain 9% of controllability is an open, connected 
subset of R 2 which contains the origin. More precisely 9$ — R 2 
as in Theorems 2 and 3 above. Furthermore, each point in R 2 
can be steered to the origin by an optimal control u (t), which 
can be assumed to be a relay control; u (t) = sgn r \2 (t). Thus the 
extremal systems are considered 

(S ± ) i = ^ 

y=-g(x)~by± 1 

Each of the systems (S-i-) has the corresponding unique critical 
point 0+ = (x+, 0), where x+ > 0, x_ < 0. Also every solution 
curve tends towards the critical point as t —>■ + oo. 

Consider the solution curve W+ 1 of S+ passing through (0, 0). 
If W+ 1 re-intersected the x axis, then the variational equation 
based on W+ 1 has a solution x (t) which vanishes twice. Thus 
consider the variational equation 

v +M+g / (x(0)tf = 0 
Let z = e bt l 2 v and compute 

Z +^g'(x(0)-L^z=0 

Now assume g' (x) ^ b 2 /4 everywhere, then z (t) and hence v (t) 
has at most one zero. Thus W+ 1 does not re-intersect the x axis 
and an easy estimate of dy/dx shows that W+ 1 is defined and is 
single valued for x > 0. A similar result holds for WJ- over the 
axis x < 0. 


Remark —In the next section an example is considered in 
which it is shown that the switching boundary is a single-valued 
curve and the switching boundary is constructed. It appears 
that the special hypothesis is easily verified experimentally. 


Example and Conclusion 


Consider the Duffing equation with control 


or the system 

(S d ) 


x+x + 2x 3 = w(Y) 
x = y 

y= -“X~2x 3 4 -u(t) 


with — 1 ^ u {t) 1. 

For the time optimal problem u (t) = + 1 on various 
intervals of time, thus consider 


Also consider the adjoint system 

Of) fi t = -^- = rj 2 (l + 6x 2 (t)) 

8 H 

' 72 = “ 8 y = -' 71 

from which is found r) 2 ( t ), that determines the interval of time 
that the solutions of 5+ and S~ are followed to obtain the 
extremal responses. 
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The switching boundary, W, is now constructed as in the 
previous theory. W+ 1 is the solution of S+ through the origin 
and lying in the fourth quadrant, see Figure 1. W- x is the solu¬ 
tion of S- through the origin and lying in the second quadrant. 

IV- 1 is now reflected along solutions of S-. From the points 
1, 2,..., 9 of fV- 1 follow backwards in time along the corre¬ 



sponding solutions of S— for a time duration equal to the interval 
between zeros of % (l)> i- e - from, say, point 1 with (0) — 1, 
»? 2 (0) = 0 follow S- and consider the response of the adjoint 
system (A) until % ( t ) is again zero. This determines a point 1 a 
on W+ 2 . This is repeated for each of the points 2, 3,..., 9 and 
gives the corresponding points 2 a, 3 a ,..., 9 a of fV+ . Since 


it is known that fV+ z is a C 1 curve, the values between 2 a, 3 a ,..., 

8 a can be filled in by extrapolation. For this reason only a finite 
number of extremal responses need be considered in generat¬ 
ing W. 

Now that W+ z has been found, W- z is the reflection through 
the origin (see Figure 1). W+ z is formed by reflecting W+ 2 along 
a conjugate interval as defined above. The remaining segments 
of W are built up in exactly the same manner. 

Thus the feedback controller u — & (x, y) for the Duffing 
equation (S a ) is found. It is very easy to store this function and 
therefore the synthesis is complete. <P(x,y) = + 1 below the 
switching boundary W and <Z> (*, y) = - 1 above the switching 

boundary W. . 

There is some question as to the accuracy with which W has 
been determined in Figure 1, but if it is nearly correct, then it is 
seen that W= IV (x) is indeed a single-valued function over the 
two flows S+ and S + . 
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DISCUSSION 


Y. Sakawa, Department of Electrical Engineering , Kyoto University, 
Kyoto, Japan 

We obtained numerically the optimum switching curve of a non-linear 
system governed by 

x+s(x 2 — l)x+x=u, £>0, |w|<^r 

by using an analogue computer, as mentioned in the paper by Hayashi 
and myself, and we recognized that when the value of e is small the 
domain of controllability covers the whole phase plane, but when the 


value of s is great the domain of controllability is bounded in the phase 
plane. In this case the condition 

— (x v ^>0 [/ (x 5 y)=£(x 2 — 1)^ + *] 

dy 

is not realized 

Did the authors obtain any general analytical results for such a 
system? 

E. B. Lee and L. Markus, in reply 

See Theorem 7 of Reference 8 of our paper for a discussion of this case. 
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Summary 

In this paper the optimal time problem is considered for multiple input 
systems when each input may be subject to a separate and possibly 
different type of constraint. 

Specifically the control of a plant described by an wth order, linear, 
time varying differential equation and which has r inputs and m out¬ 
puts is considered. Each input u^t) is constrained by a condition of the 
form 

[ So | u M Pi dt] « g L { , for i = 1,2,...,r 

wher opi > 1. Note that the constraints are applied to each input sep¬ 
arately so that this problem includes those practical cases when some 
inputs may be, limited in magnitude, some in energy, and others in still 
different ways. The prqblem is to find the set of inputs u t (t) which 
satisfy the constraints and which will make the plant outputs equal to 
a set of desired outputs in minimum time T. 

The technique presented in this paper for solving the problem is an 
extension of the methods of functional analysis first used by Krasovskii 
and Kulikowski to solve optimal time problems with a single con¬ 
strained input. A useful feature of the solution presented in this paper 
is that a very general norm is defined for the input vector u(t) — 

..., u r (t)] which incorporates all the separate constraints 
into a single constraint condition. This permits a solution of the opti¬ 
mal time problem for this case, which may be obtained with no more 
formal difficulty than in the single input case. 

Sommaire 

Cette communication a trait au probleme du temps optimal pour des 
systemes a entrees multiples dans lesquels chaque entree peut etre 
separement l’objet d’un type donne de contrainte, eventuellement 
different pour chacune d’elles. 

Elle s’attache en particulier au reglage d’une installation qui est 
regie par une equation diffdrentielle lineaire d’ordre n par rapport au 
temps et qui possede r entrees et m sorties. Chaque entree u^it) est 
soumise a une contrainte qui s’exprime par une condition de la forme 

[So| u i(0 | A * <*/] ** ^ pour i = 1,2,... ,r 

°u Pi ^ h II convient de remarquer que ces contraintes sont appli- 
quees separement et chaque entree, de sorte que le probleme ainsi 
pose inclut les cas pratiques ou certaines entrees peuvent etre limitees 
en amplitude, certaines en energie et d’autres de maniere encore 
differente. Le probleme consiste a trouver l’ensemble des entrees u^t) 
qui, tout en satisfaisant les conditions representant les contraintes, 
rende les sorties egales a Tensemble des sorties desirees, et cela en 
un temps minimal T. 

La technique developpee dans cette communication pour resoudre 
ce probleme est une extension des methodes d’analyse fonctionnelle 
appliquees pour la premiere fois par Krasovskii et Kulikowski a la 
resolution des problemes de temps optimal dans lesquels il n’y a 
qu’une seule entree sujette & contrainte. Un des avantages offerts par 
la methode presence dans cette communication reside dans le fait 
qu’elle permet de definir une norme tres generate pour le vecteur 
d entree u(t)— [u x (t\ u 2 (t\ ..., u r (t)] ce qui permet d’englober en 

NSF Q^45j4 Search Was su PP° rted unde r AF-AFOSR-62-144 and 


une unique condition de contrainte toutes les contraintes individuel- 
les. D’ou la possibility d’obtenir pour ce cas une solution du probleme 
du temps optimal, et cela sans plus de difficulty formelle que pour le 
cas de l’entree unique. 

Zusammenfassung 

Der Aufsatz betrachtet das Problem der zeitoptimalen Systeme mit 
mehreren Eingangen, wobei jeder Eingang einer getrennten und mog- 
licherweise unterschiedlichen Art von Beschrankungen (Bedingungen) 
unterliegen kann. 

Im besonderen wird die Regelung einer Strecke mit r Eingangen und 
m Ausgangen betrachtet, die sich durch eine lineare Differentialglei- 
chung //-ter Ordnung mit zeitvariablen Koeffizienten beschreiben laBt. 
Jeder Eingang u t (t) unterliegt einer Bedingung der folgenden Form: 

UJM" d/]* ^ fur /= 1,2,...,r 

wobei pi > 1 ist. Da diese Bedingungen fiir jeden Eingang einzeln gel- 
ten, schlieBt das hier behandelte Problem auch die praktischen Falle 
mit ein, bei denen in einigen Eingangen der Betrag, in einigen die Ener¬ 
gie, und andere in ganz anderer Weise begrenzt sein konnen. Es geht 
darum, die Gruppe von Eingangen u^t) zu finden, die den Beschran¬ 
kungen geniigt, und die in einer minimalen Zeit T die RegelgroBe auf 
die gewiinschten Werte bringt. 

Die hier zur Losung des Problems benutzte Methode ist eine Er- 
weiterung der Methoden der Funktionalanalysis, wie sie zuerst Kra¬ 
sovskii und Kulikowski fur zeitoptimale Systeme verwendeten, bei 
denen nur ein Eingang einer Beschrankung unterliegt. Eine niitzliche 
Eigenschaft der hier vorgelegten Losung liegt in der Definition einer 
sehr allgemeinen Norm fur den Eingangsvektor u(t) = [u x (t), u^t), ..., 
u r (t)], die alle getrennten Bedingungen in einer einzigen erfaBt. Hier- 
durch laBt sich das vorliegende zeitoptimale Problem ohne zusatzliche 
formelle Schwierigkeiten wie im Falle eines Systems mit einem Eingang 
losen. 


Introduction 

In a recent paper 1 , the authors showed how Kulikowski’s 2 * 3 
application of functional analysis techniques to the solution 
of optimal control problems could be extended to multiple input 
systems. This was accomplished by defining a general norm on 
the entire input vector so that constraint conditions could be 
applied to all the inputs simultaneously. That paper, however, 
was concerned with problems in which all inputs have the same 
type of constraint (e.g., when all inputs are limited in magnitude 
or all are limited in power or energy), and the constraint is 
imposed on the totality of the inputs, (e.g., when the combined 
power dissipated by all inputs is limited). 

In the present paper, a similar procedure is applied to a more 
general problem. In this case a problem is considered, in which 
each input is individually constrained and where the individual 
constraints need not be of the same type. This problem is 
approached by first embedding it in a still more general problem 
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in which a very general norm constraint is applied on the entire 
input vector. It will then be easy to show that the solution to the 
actual problem as well as to the problems considered elsewhere 1 ’ 2 
are just special cases of this general solution. 


Statement of the Problem 


This paper is concerned with the solution of time optimal 
problems when separate constraints are imposed on each input. 
Given: 

(a) A completely output controllable plant (see Appendix I) 
described by the linear differential equations 

x(t) = F(t)x(t) + D(t)u(t) (1) 

with output y (t) = M (/) x (t), where the state of the system 
at any time t is represented by the n-dimensional vector x (t); 
the r control inputs are represented by the r-dimensional vector 
n ( t ); the n x n matrix F (/), the n x r matrix D (t) and the 
m x n matrix M (t) are called the system matrix, distribution 
matrix and output matrix respectively; 

(b) The initial state of the plant at time / 0 

x (to ) ==x 0 


(c) A desired output y d (t); 

(d) A separate constraint on each plant input Ui (t) in the form 


IIm* 



( 2 ) 


where > 1 for i = 1, 2, ... r. 

(Note that in this problem the p t and Li may be different for 
each i. This means that each component of the control vector 
(each control input) may have a different type of constraint. 
For example, if the input u 3 - is to be constrained in magnitude 
then p 3 = °o; if it is to be constrained in energy then/?* = 2, etc.) 

The problem is to find that control u 0 (t) which satisfies the 
constraints and makes the plant output y at time t x equal to the 
desired output (i. e., y (t x ) = y d (t x )) for minimum elapsed 
time T defined by T = t x — t 0 > 0. 


A Simplified Problem 


It is instructive to first consider a simplified version of the 
general problem. Assume a time invariant, linear plant which 
can be activated by two possible inputs u ± (t) and u 2 (0> and 
which has a single output. If x (t) is the response (the total 
output consists of the response x ( t ) and the response due to 
initial conditions which is assumed to be known) of this plant 
to the signals u x ( t ) and u 2 (0 which are applied at t == 0 then 


x(t) = 


k 1 (f —t) u x (t) dt + 


k 2 (t — T)u 2 (V)dT (3) 


where k x (t) and /c 2 (t) are appropriate impulse responses. 

The problem is to determine u x (/) and w 2 if) so that the 
response x ( t) should reach a desired value X in minimum time 
t — T 0 subject to the constraint || u x \\p x < L x and || u 2 1| p 2 < L 2 
[see eqn (2)]. 

First, let x (t) = X at some given time T. Then 


X=x(T)=x 1 (T) + x 2 (T ) (4) 


where 

and 




x 1 ( T) = Jo 


k 1 (T — z)u 1 ( T)dt 



( T-z)u 2 (t) dr 


(5) 

( 6 ) 


It follows from Holder’s inequality [see Appendix II, eqn (84)] 
applied to eqns (5) and (6) that 


MT> | = | 


k 1 (T-'t)u 1 (i:)dT\<\\k 1 \\ il \\u 1 \\ l , 


and 


|x 2 (T) | = | ^2 (T-r) u 2 (t)dt| < ||fc 2 |l« ll« 2 ll„ 

o 


where 




-. T 
Jo ,fe ‘ 




1/41 


( 7 ) 

( 8 ) 
(9) 


and 1 /pi -1- 1 fa = 1. 

Simplify the notation denoting || u. t || pi and || || qt by (| || 

and || ki || respectively. By making use of the constraints on 
Ui (/) and eqns (7) and (8) 


and 


l*t(T)l 

IIM 


<||ui||<Li 


( 10 ) 


IX-X^DI 

\\k 2 \\ 


1*2 CO I 

IIM 


<\\u 2 \\<L 2 


( 11 ) 


or rearranging the above equations, 


L 1 ||fe 1 ||>|x 1 (T)| (12) 

and 

L 2 ||/c 2 ||>|X-x 1 (T)| (13) 

The smallest value of T for which both inequalities (12) and (13) 
are satisfied is the required optimal time T 0 . 

Assume now that for some fixed value of x x (T), say x v there 
is a range of values of T for which both eqns (12) and (13) are 
satisfied. When x x = 0, this occurs when T>T X (see Figure 1); 
when x t # 0 and sgn x x = sgn X , inequalities (12) and (13) are 
both satisfied when T x > T> T 0 . T 0 is then the optimal time and 
it can be found by requiring that both inequalities (12) and (13) 
are satisfied simultaneously with a sign of equality when 
sgn x x = sgn X . (Note: when x L = x x # 0 and sgn x x ^ sgn X, 
then inequalities (12) and (13) may be also satisfied as shown 
in Figure 1 for some T'>T 2 but the minimum value of T 2 is 
always greater than T x and therefore also greater than T 0 .) 


L.IM 
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Therefore, 

U IIM = |x?| 

(14) 

and 

L 2 \\k 2 \\ = |X-x?| 

(15) 

and also 

x i (T 0 )=x°=L 1 llikjll sgn A - 

(16) 

Using a similar argument as above it is found that 



(r 0 )=X - x° x = L 2 II k 2 II sgn X 

(17) 


Assume now that eqns(14) and (15) are satisfied. This 
implies that inequalities (10) and (11) and therefore also eqns (7) 
and (8) must be satisfied with a sign of equality. By Holder’s 
inequality [see Appendix II, eqns (84) and (85)] applied to 
eqns (5) and (6) this occurs if 


Ul(T)= (1 |fcl(r_T)r_1 sgn ( lg ) 

and 

Xo(T) 

112 T lkz (T ~ r)r ~ 1 sgn k >< 19 ) 

Optimal inputs Uj 0 and w 2 ° are obtained by substituting T = T 0 
in eqns (18) and (19) and by making use of eqns (16) and (17) 
Thus 

«?(t)=L 1 (||fc 1 || 9l ) 1 -«> |/c 1 (r 0 -T)p- 1 (20 ) 

and 

u °2 ft)=L a dl^llj 1 " 92 1 k 2 (r 0 - T)p- 1 sgn MEfdl ( 21 ) 

A 

To simplify the calculation of T 0> note, making use of eqns (14) 
and (15), that 

Li ll^xll+Lj ||fc 2 || = l^xl + |AT-xJ|^|Z| (22) 

Clearly then, T 0 is the least value of T which satisfies the equation 


L i\\K\\ qi +L 2 \\k 2 \\ q =\X\ (23) 

In fact, if eqn (23) cannot be satisfied for any T, then there is 
no solution to the problem. 

Making use of eqn (23), eqns (20) and (21) can be written 


“iW= 

and 




«z(?) = 


II*iII 81 +l 2 ||fc 2 ||jj 


1 sgn k 1 (T 0 — t) 
(24) 

1^2 (To 1 sgn k 2 (T 0 — t) 


It can be verified by direct substitution that the inputs given in 
eqns (24) and (25) satisfy eqn (3) for any time t = T (not 
necessarily for T = T 0 ) and that * (T) = X, the desired value. 
At the same time it can also be verified that 


IKIL= 



ll“2lU 


IIM 42 


l*zl 
II ^2 II 42 


(26) 

(27) 


II i Up, _ L x 

\\u 2 \\ p ~T 2 


(28) 


It follows from inequalities (10) and (11) that eqns (26) and (27) 
give minimum values for || u t || Pi . Thus it is seen that the time 
optimal problem could have been approached by first trying 
to find «! (t) and u 2 (r) which satisfy the terminal condition 
x(T) = X with minimum values || u { || p . and also the additional 
constraint of eqn (28). 


A General Norm 5 


Before proceeding to the general solution of the problem it 
will be convenient at this point to consider a means of represent¬ 
ing the set of r constraints (2) as a single constraint condition. 
This is readily done by defining a norm of u as 


||«|| = max [llwdlpj/Lj (29) 

Note that when the single condition 


MI<1 (30) 

is satisfied, then it is apparent that the r conditions given by 
eqn (2) are also satisfied. This means that the v input constraints 
can be imposed by requiring that the single condition eqn (30) 
be satisfied. Unfortunately, the norm defined in eqn (29) is a 
difficult one to work with directly; a simplification in the 
solution is achieved by a further embedding of the constraint 
conditions in a still more general form of norm on u (t). 

This more general norm can be defined by 


l«ll P = 



L7 P \\Ui\ 


Pi 


i Ip 


(31) 


where p > 1 and || u ( || JJi is defined in eqn (2). Using the same 
method as Kirillova 6 it can be shown that in the limit as p -> oo 
the solution with || u \\ p constrained approaches the solution 
with || « || constrained, so that the constraints of eqn (2) will 
be replaced by the constraint 


NI P :£l (32) 

The solution of the originally stated problem is thus obtained 
by first solving the more general problem which has a constraint 
given by eqn (32) and then letting p-+oo. 

It may be of interest to point out that the problem considered 
by Kranc and Sarachik 1 is a special case of this more general 
problem. This fact is easily verified by setting all p { = p and all 
Li = L so that condition (32) gives 


IN 



KOOI^dT 


it/? 

<i 


(33) 


The problem considered 1 was the same as that stated here except 
that the constraints were those of eqn (33) instead of eqn (2). 


Solution of the Problem 

In order to solve the stated problem, utilizing the approach 
ul" Krasovskii 7 , consider first the following alternate problem; 

Given a plant, initial condition, and desired output as 
described in Problem Statement (a), (6) and (c). 

Find that input u (t) which at a fixed time t x malr«j j? (Q = 
y 4 (q) and which has a minimum norm || u ||j,. 
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Note that in this alternate problem the time at which the 
actual plant output must equal the desired output is fixed at the 
start and no constraints are imposed. The minimum time solution 
will be obtained from this ‘minimum norm’ solution by choosing 
the smallest time for which the minimum norm || u \\ v is just 
equal to unity. 

The solution of the differential equations describing the 
plant (1) which satisfy the given initial conditions x 0 at time t 0 is 


and since f A (*) is a linear functional, if any linear combination 
of the h j (t, t) is considered, such as 

m 

Z Xjhj(t u x)=XH(t u x) (43) 

j = i 

where A is an ^-dimensional row vector, then by linearity and 
eqns (42) and (43) 


x(t) = <P(t,t 0 ) x 0 + # (f, t) D (t) u (t) dx (34) 

J fo 

where 0 ( t , t) is a fundamental matrix 8 of the plant which also 
satisfies 0 ( t , t) = / and will be called the transition matrix. 

The output is given by 


(44) 

for any k (t l9 r) which can be represented by eqn (43). 

Defining the quantities 

r ft* li iQi 

iifcfL- Mt^rdxl (45) 


y(t) = M(t)$(t,t 0 )x 0 +\ M (f) $(t,x) D(t) u (t) dr (35) 

J t 0 

These equations become simpler if the following definitions are 
introduced 

e(t)=y(t)-M(t)$(t,t 0 )x 0 (36) 

and 

H(t,x)±M(t)$(t,x)D(x) (37) 

Note that the m X r matrix H (t , r) is the impulse response 
matrix of the system and e ( t ) represents the difference between 
the actual output and the output which would have been 
obtained from the initial condition alone if no inputs had been 
applied for t > t 0 . 

Using eqns (36) and (37), eqn (35) becomes 


= Siiiiyj, 

.J=i 


where k { (t l9 r) is the zth component of the row vector k (t l9 r) 
defined in eqn (43) and q t and q are related to p t and p of eqn (31) 
by 1 lq t + 1 IPi = 1 and 1 Ip + 1 \q == 1. 

Now consider the quantity || fA || (called the norm of the 
functional /a) defined by 

“-T-flf} (47) 


e (t) = H (t, t) u (t) dt (3 8) 

J to 

The problem now is to find the input vector u (r) with min || u 1^ 
which satisfies 




e d (t 1 )=y i (t 1 )-M(,t 1 )$(t u t 0 )x 0 

or equivalently to find the input vector which satisfies the m 
equations 

% t t 

h j (t 1 ,x)u(x)dx=e d j(t l ) for j=l,2,...,m (40) 

J to 

where hj (t, r) is the ./th row of the matrix H (t, t) and e$ d (t ± ) 
is the jth element of the vector e d (t x ). 

Since the plant is completely output-controllable it is known 
that at least one input exists which meets these terminal con¬ 
ditions for some t x and the one with minimum || u is sought. 

Now denote by U A the set of all inputs which satisfy eqn (40). 
Define the functional f A (hj) by 


[ \^\h)\ \ 

. f 1 \ 

J m \ 

III I wa\ 

7 = 1 

— lllaA l | 

ie<,=1 III E AAH.I 

J = 1 


where k is given by eqn (43). Using eqns (43) and (44), eqn (47) 
becomes 


||/J=max 


Denote by A* the vector whose components 

m 

minimi/ e, || 2 h II subject to the condition k e d (r x ) = 1 . 
j — 1 

Therefore 

m m 

min \\T,W\ t =\\l^ h A-W k X f49 ) 

~1 • / " 1 J 1 

and eqn (48) can be written as 


Considering now the functional 


L. (/*;)= I hj(t u x)u(T)dT (41) 

J to 

where u (r) belongs to U A (u (r) e Ua)- Therefore 

/a (hj) = e j ( f i) ^ 42) 


/*(*•)« k*(t u x)u(x)dx (51) 

J to 

and applying the generalization of Holder’s inequality derived 
in Appendix II it is found that 

|/ 4 (fc*)!<||fc*||J|K||p (52) 

or by eqn (50) and because f A (k*) = A* e d (f,) = 1, that 

II II ^ I/a (fc*)l __ ^ _ li f li ( 53 ) 

p5: ||k*ll, II**II, fA ( ) 

Inequality (53) states the necessary condition that u (x) e Ua- 
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In addition, from eqn (53) is obtained the necessary condition 
that u (t) e U A with a min norm || u l^, namely 

W ' = wir l/J (54) 

The above condition will be satisfied whenever inequality (53) 
and therefore inequality (52) are satisfied with the sign of 
equality. By Appendix II, when the vector u (r) has its elements 
u t (r) of the form 

(r) = KLi \\k*\\\; qi \k*{h, sgn kf (t u t) (55) 


then the sign of equality holds for eqn (52). 

Substituting eqn (55) into (54) the constant K is evaluated 
to be 


K = - 


(11*1/ 


(56) 


Thus, eqn (54) is satisfied if and only if 

Lf 


«i(0= 


(\\k%y 


ll^f ll|r^ £ l/cf t)|^- 1 sgn fcf (r l9 t) (57) 


It follows from the above argument that the necessary condition 
that u eU A with a minimum norm || u \\ P is that u t (r) be of the 
form dictated by eqn (57). To show the sufficiency of this pro¬ 
position it must be demonstrated that the signal u (r) given by 
eqn (57) is member of U A in that it satisfies eqn (40). This is 
done in Appendix III using an indirect argument. 

As was mentioned earlier, the constraints of eqn (2) on the 
input components in the original problem can be imposed 
merely by considering the norm || u || as defined in eqn (29). 
In accordance with this earlier discussion the input is found 
which has minimum norm [| «* || and which satisfies the condi¬ 
tion y (t x ) — y d (t x ) by just setting q — 1 (this corresponds to 
p — oo) in eqn (57). In this case it is found that 

Ui(tf= Wh. llknlq ~ qtlk * itu0|€i_1 sgnkt ( 58 > 


where the starred quantities are obtained by finding 2* which 
gives 

11**11 i =min £ m\ki\\ qi (59) 

A i= 1 

subject to the constraint that k e d (t x ) = 1. 

It should be noted that the actual minimization required 
by eqn (59) is much more difficult than for single norm con¬ 
straints. 

To obtain the time optimal solution required by the original 
statement of the problem the effect of the constraints must be 
considered. It has been shown that all the constraints are 
satisfied by the single condition 


NI<1 (60) 

Now from eqn (52) it is seen that for p = co, q = 1 

<«> 

This shows that for any a solution is possible if and only if 

11**11! >1 (62) 


When || k* || x is a continuous function of 4 the minimum 
elapsed time T 0 — t x * — / 0 is achieved for the smallest t x ~ tf 
which makes 

ll**lli = l (63) 

This equation is used to find 4*. Now if the minimum norm 
solution of eqn (58) is used then for t x = tf it is seen that 

l|H * ll= l*k =1 (64) 

so the solution given by eqn (58) satisfies the constraints and 
the end conditions for minimum elapsed time and is therefore 
the time-optimal solution. Using eqn (64) this gives 

u i (t) opt =L i ||fcf ||i~ 4, l*?(*i. OP -1 sgn kf (f* t ) (65) 

where kf is obtained by solving eqn (59) for A* and substituting 
this value into eqn (43). Eqn (63) must then be solved for 
minimum t x which gives tf. 

It can be verified by forming || w* (/) opt IL from eqn (2) 
for the input given by eqn (65) that || w £ (f) op t \\ pi = L i for all /. 
This shows that the optimal time solution is given by an input 
for which all constraints are satisfied with an equality. 

If || k* ||j is not a continuous function of t x the necessary 
and sufficient condition for the existence of a solution is still 
eqn (62). In this case, however, a true minimum may not exist, 
although the greatest lower bound (infimum) may be approached 
arbitrarily closely. 


Numerical Example 

To illustrate how the preceding results may be applied, 
a very simple example will be considered taking the plant 
shown in Figure 2. Its differential equations written in normal 
form are 

*i(0 

*2(0 

-*3(0. 


0 1 o' 


X 


0 

0 



0 0 1 


*2(0 

+ 

0 1 


«i(0 

0 

0 

0 


X*(t) 


1 0 


|_ W 2 (OJ 


( 66 ) 



Figure 2. Plant for numerical example 


This system is time invariant and its fundamental matrix 
$ (t — 1 0) is obtained from the solution of the unforced equa¬ 
tions which satisfy the initial condition x (/ 0 ) = x 0 . For invariant 
systems the easiest method of obtaining the fundamental 
matrix (j) (t — t 0 ) is by using Laplace transforms. For a system 
described by 

x = Fx + Du 

0 ( t) is found from 

0(r)= c Sr 1 {[s/»F]" 1 } 

where 3?- 1 means inverse Laplace transformation. 
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Applying this procedure to the system in eqn (66) gives 

Taking the inverse <5? transform of each element yields 



"s -1 o' 

-1 

s 2 s 1 

1 _ 

0 s — 1 

1 

— 3 

0 s 2 s 



s 



o 

o 

!_ 


_0 0 s 2 _ 


*( 0 = 


i t 


r 


2 

0 1 t 

0 0 1 


(67) 


To make the example specific say that it is desired to find 
the inputs jq (/) and u 2 (t) which will take the system from a 
given initial state to the origin jc'* = 0 in minimum time. The 
input « x (/) is constrained in energy (|| u x || 2 < L x ) and the input 
h 2 (?) is constrained in amplitude (|| u 2 1|< L 2 ). Note that in this 
case since the output y (J ) = x (?) which is the state vector itself, 
M in eqn (1) is the identity matrix. 

From eqn (37) H (q — r) is found as 




2 

t x — x 1 

.1 0 

so the row vectors h x (? : — r), h 2 (q — r), h 3 (q — t) are 

(fi-'t ) 2 


1 1 — T 


( 68 ) 


fcx(q-t)= 


,t i-T 


(q Fi t, lj 

J*3(*i-T> = [1,0] 

The row vector k (q - r) obtained from eqn (43) is 

fcpi-i 0 = E ^-(q-t) 
j = 1 


(69) 


Ax 


(tx— t ) 2 + A 2 (q — t)+A 3 , Ax (q q)+A 2 


Now since p x = 2 and p 2 = rfJ , there is q x — 2 and 1 
which gives 


PilU= 

P 2 IU= 


Ax 


(q-r) 2 +A 2 (q-T)+A 3 


dr 


1/2 


(70) 


| Ax (q —t) + A 2 | dr 

linimize 

||fc||x=Lx ||fcx || 2 + L 2 ||fe 2 |li 


LJ (o 

It is desired to minimize 


(71) 


subject to the constraint A e d (q) = £ A 3 - ef (q) = 1. Saying 

2 = 1 


that the initial condition x 0 
gives 


which for y d (q) = x <i (q) = 0 


c 4 (q)=/ (q) — M<Hq ~ *o) *0 = 


(72) 


(Note: This particular x 0 gives (q) which does not depend on 
q; in general, however, e d (q) will be a function of q.) 

Now, putting rj = q - r, T — t 1 — q> and using eqns (70) 
and (72) it is found that A 2 = 1 and the minimization of eqn (71) 
becomes 


min L x 

A 3 


LJ 0 


T / 2 \ 2 

) drj 


” 1/2 rr 

J +Lj j. 


|j/ + A 2 1 d?j 


Taking partials with respect to A 2 and A 3 and equating to zero 
gives 

nr T 2 

A$= — =r- and A*=- 


and therefore 


l|k*lli=-Li 


T 5 \ 

720/ 


1/2 


+ L 2 


3 ‘ 12 
rj-i 2 


(73) 
zero 

(74) 

(75) 


For any constraints Lx and L 2 the minimum time interval T 0 is 
obtained by plotting || k* || x in eqn (75) and finding T which 
makes it equal to unity. 

The optimum inputs will be 


and 


PetllT 0 ” 


tf 


rrt np2 


« 2 ( 0o P t=i2Sgn 



(76) 

(77) 


for ?„</< q + T o where || /ex* || a in eqn (76) is a number ob¬ 
tained by using Aj = 1, eqn (74) and T 0 in eqn (70). 


Conclusions 

The procedure presented in this paper shows how to obtain 
the optimum time solution for a system with multiple input 
constraints. The form of the solution is readily obtained, but 
it should be noted that the actual evaluation of the optimum 
input requires that one be able to find A* which gives the mini¬ 
mum of || k Hi. This in itself may be a difficult numerical problem. 
The procedures suggested by Kulikowski 2 ’ 3 which make use of 
known minimizing properties of Legendre or Tchebyscheff 
polynomials and which may be used in the special cases of 
energy or amplitude constraints respectively, for systems with 
a single input or with multiple inputs constrained in the same 
way (i. e. Pi — = ... — p r — p ) are not applicable to the 

minimization of || k 1^ as given by eqn (59). When variational 
methods 10 or the maximum principle of Pontryagin 11 are used 
to solve optimization problems one finds that a subsidiary 
mathematical problem also arises. In these cases the sub¬ 
sidiary problem is a two point boundary value problem which is 
not well suited to a computer solution. On the other hand, when 
the approach presented in this paper is used, the minimization 
problem which arises is well adapted to solution by a computer. 

Appendix I, Controllability 

The following definition is equivalent to the definition offered 
by Kalman 4 . 

Definition 1—A plant is said to be completely controllable 
if there exists a control input u (t) which will move the plant 
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from any initial state x (/ 0 ) at any initial time t 0 to any desired 
state in a finite time interval [> 0 , fj. 

Definition 2 —A plant is said to be completely output con¬ 
trollable if for any time t 0 and any state x (t 0 ) there exists a 
control input u (t) which will move the plant initially in x (t 0 ) 
to any desired output in a finite time interval. 

It can be shown in a manner identical to that of Kalman 4 , 
that a necessary and sufficient condition that the system described 
by eqn (1) be completely output controllable, is that the matrix 


r(t 0 ,t 1 )=M(t 1 y 


$(t 1 ,t)D(t)D T (t)^ T (t 1 ,t)dt\M T (h) 
to J (78) 


be positive definite for some finite time t 1 > t 0 . (The superscript T 
denotes matrix transpose.) Furthermore, if the system of 
eqn (1) is time invariant, then a necessary and sufficient condition 
for complete output controllability is that 


rank {M[DIFD = m 


(79) 


Note that when condition (78) or (79) is satisfied for M = / 
(the identity matrix) then the system is completely controllable. 
Also note that complete controllability implies but is not 
implied by complete output controllability. Plants considered in 
this paper are required to satisfy the weaker condition of com¬ 
plete output controllability. 


Appendix II, A Generalization of Holder’s Inequality 


The purpose of this appendix is to obtain an extension of 
Holder’s inequality to an integral of the form 


x(t)-y(t)dt = 
In the following it is shown that 
(t)-y(t)dt 


b n 


E *i(0y;(0df (80) 

a i— 1 


I" n "1 l/pf™ n 'll /q 

x(t)-y(t)dt < YLl'WXiKt lL\\\y t r qt (81) 
J a Li = l JLi = l _ 


for p >. 1, q > 1 and 1/p + 1/q = 1 where | Xi ( t ) \ p* and 
| (t)\ q i are integrable, the are positive constants, and where 



for Pi > 1, q A > 1 and l/pi + 1/^ = 1. It is also shown that 
the inequality in eqn (81) becomes an equality if, and only if 

Xi(t) = KL\ ll^lljr^l^wr'^gn y,(i) (83) 


for all a < t < b and all z = 1,2, ... n where K is an arbitrary 
constant. 

From the usual form of the Holder inequality as applied to 
integrals 5 it is known that 


If* r f * li fpf Tb ll /q 

x(t)y(t)dt£ |x(l)| p df \y(t)\ q dt (84) 

do LJ 0 J LJ a _ 


for p > 1, q > 1, and l/p + 1/^ = 1 when | x (Ol 3 " and | y (/)|« 
are integrable, and that this inequality becomes an equality if 
and only if 


x(t)=K\y(t)\ q ‘sgnjO) (85) 


where K is an arbitrary constant. These results can also be shown 
to hold 1 ’ 9 when p = 1 or q — 1 if condition (85) is properly 
interpreted. 

The equivalent Holder inequality for sums 5 is 


n r n ll/pf n “ll lq 

E ^ E E Pt 

i=l Li= 1 J Li= 1 _ 


where the inequality holds if and only if 


( 86 ) 


a—KI&l 3 i sgnp i for z = l,2, ...,n (87) 


where K is an arbitrary constant. 

Again, these results are usually shown to hold for 1/p + l/q 
= 1 ,p > 1 and q > 1, but by proper interpretation of eqn (87) 
they can be shown to hold also for p = 1 or q = 1 1 » 9 . 

To obtain a Holder inequality for the integral of eqn (80) 
observe that 


b n 

E x i(0ki0)df 

J a i= 1 



Xi(t)yi(0dt 


( 88 ) 


and that the equality in eqn (88) holds if and only if 


Xi (0 yt(t)>0 [or x ; (0 y i ( t )<0] 


(89) 


for a < t < b and all i = 1, 2,... n. 

Now applying the results of the simple Holder inequality 
as expressed in eqn (84) it is found that 


r 


Xi(t)yi(t)dt\ 


iSlMU llttll* 


(90) 


for pi > 1, q { > 1 and 1 /p t + 1 /q t = 1 when | x t (0l P£ and 
| y t (t)\zt are integrable, where || \\ Pi and || y t \\ qi are defined by 
eqn (82). By eqn (85) it is found that the inequality in eqn (90) 
becomes an equality if and only if 


x i (f) = i^ i |>’£(0l 4i_1 sgnj; i (0 for a<t<b 
Inserting eqns (80) and (90) into eqn (88) gives 

'b 


l 


*( 0 ‘T( 0 d * 


^E IMUttll, 


i= 1 


(91) 

(92) 


Note that if eqn (91) holds for all i = 1, 2,... n and if all K t 
have the same sign then conditions (89) are satisfied. This means 
that the equality in eqn (92) is valid if and only if eqn (91) holds 
for all i = 1, 2, ... n and all have the same sign. 

Now if 

* _ ii**iu 

£ L, (93) 

and 

Pi^Li ||y,|| 4I 


where the Li are positive constants, then since and % are 
positive it is found that 


E 


*iPt= 


E 


i= 1 


(94) 
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and using the results of eqns (86) and (87) it is found that satisfies the terminal conditions 


n I r n li/p r n li la 

L?i lj)il J (95) 

where p > 1, q > 1 and lIp +■ \\q = 1, and that the equality 
holds if and only if 

Jt|j> £ I ( ^" 1) sgn j> £ for 1 ( 96 ) 

where J? is an arbitrary positive constant. However, since x ?; and 
are positive, eqn (96) can be expressed as 


( 97 > 

where £ is a positive constant. When eqn (93) is inserted in 
eqn (95) and the resulting equation combined with eqn (92) gives 


'5 

x (t)-y(t) d t 

a 


< 


zwwi 

; = i 




1 /fl 


which is the desired inequality (81). The equality in eqn (81) is 
satisfied if and only if the conditions for the equality in eqns (92) 
and (95) hold. This requires that 

(a) all Ki have the same sign; (98) 

(b) eqns (91) and (97) hold. 


Considering condition (97) it is found that 


^L^Wx^LJ 1 


~~ rb 
_J a 


1/Pi 


:xiO)rdf 

=jtj)r i_) =^ ( r 1 ) Mr 1 (") 


^(«i)= h j {t l ,x)u(x)dx 

J to 

for j = 1,2 ,m, consider the problem of finding 

J/ 


(104 


max 

X 


X i) 
1=1 _ 


PL 


(105) 


To do this take 
J 


0 


Z 

J = 1 


. B*ll, 

11*11, sgn 

sW» 

i_i 

1 

he d j(h) 

e%h) ~ 

m 

Z 

J = 1 

■£rwu 

<u * -n 


> 


which gives 


a 


for 1 = 1,2, ...,m (106) 

(107) 


From the definition of || k || 9 in eqn (46) it is found 

£-ii*n,-kii*u 1 " 4 z dog) 


QX t 


i= 1 


Now insert Xi (/) as given by eqn (91) in the integral of eqn (99). 
This gives 

L7 1 ix,i ii^ii;! / w = xl\~ 1 ii^ii;; 1 (100) 

Solving for | Ki | gives 

\Ki\ = £L q i \\y i r q ; q ‘ for all i=l,2 ,n (101) 

Therefore if K ( in eqn (91) is always chosen so that eqn (101) 
holds then eqn (97) will also be satisfied. It should also be noted 
that Ki may be positive or negative provided that all K { have 
the same sign. This means that eqns (101) and (98) can be 
replaced by 

for all i=l,2,...,n (102) 

where K is an arbitrary constant. 

Substituting this result into eqn (91) gives the final form of 
the necessary and sufficient conditions for an equality in 
eqn (81) as 

x t {t)=KL\ IMOr 1 sgn J>,(0 

for a <, t <, b and all i = 1, 2,... n. This is the desired result 
eqn (83). 

Appendix HI 

In order to show that the input u (r) with components given by 
u i( T )=^7T« HtfllSr*' Ikfih^T-^gn kf (t u t) (103) 

(II* II ,) 9 

for t 0 <x<t x 


and in turn 

^■o*.u , -4ii^i;r 1 ^ii fe * || « ( 109 ) 

From the definition of || ||^ (eqn 45) 

4-OI^HJ 

rti t) 

= [||fc 1 iy 1 - 41 |fcj(<i,T)l* 1 s §n k i (t u x) ‘ 8A ’. -dr 

J to l 

( 110 ) 

From eqn (43) it is found that k t (q, r) is given by 

m 

k i (t 1 ,r)= Yj Ajhij(t i , t ) ( 111 ) 

i=i 

where h i} (q, r) is the element in the ith row and;th column of 
the matrix H (q, r). This equation shows that 


9*i0i>x) 

3Aj 


= *u (t u x) 


( 112 ) 


Substituting eqn (112) into (110), eqn (110) into (109), eqn (109) 
into (108) and eqn (108) into (107) gives 


Z V;(*t) 

e?(*i)=^rn- W\l' q 


11 * 11 , 


Z Will 


1=1 


q-cn 

3i 


|fci(*i, x)l 9i - 1 Sgn ki(t u t) h,,(t u t) dr (113) 
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This equation can be rewritten as 

e?Oi)=| E h(t U T) 

J to i= 1 

m 

E Vj( ? i) 


j = 1 


11*112 


-i?ii*iiir* , i*«ai,Tr- i sgnfe J (t 1 ,T) 


di 


for l— 1,2, ...,m (114) 


Now when the set of eqns (114) are solved for Ay’s these values 
of Ay will give the maximum in eqn (105). 

Alternatively if a set of Ay’s which maximize the expression 
in ,eqn (105) are inserted into eqn (114) then these equations are 
satisfied identically. Recall that the values Ay* defined after 
eqn (48) makes (105) a maximum. Therefore when Ay*’s are 
used in eqn (114) and the quantity in the brackets is denoted 
as (t) this is precisely the form of u t (r) given in eqn (103). 
It can be concluded therefore that when wy (tt) is chosen as in 
eqn (103) the set of eqns (114) become 


ti r 


e i(h) = Z h il (t li z)u i (x)dT = 

't Q i = 1 


dr 


for Z = l,2,..., m (115) 


and are satisfied identically. These equations are seen to be the 
terminal conditions (104). 
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The Approximate Calculation of a Class of Automatic 
Systems with Forced Parameter Optimization 

Yu. I. ALIMOV 


Summary 

The paper considers: (1) two linear filters <fr x and d> 2 connected in 
parallel, with an input signal 6 (7), and (2) closed-loop systems for 
adjusting the parameters Xt of filter <& 2 , which employ a search 
modulation ftxt ( t ) of these parameters. The self-adjustment network 
includes a detector (linear or square-law) for the output quantity of 
the filter fl? s — <D 1} phase discriminators and averaging filters 

On the assumption that the amplitudes of the signals dxi (t) are 
small, approximate (and, within the limits of the small-parameter 
method, entirely correct) expressions are found for the control actions 
in the self-adjustment networks in terms of the frequency character¬ 
istics of the filters d} x and <I> 2 and the present spectra of the signals 
<5 xi(t) and 6 (t). The differential equations derived for the self¬ 
adjustment processes take account of the limited memory and pass- 
band of the filters <1^, <t> a and W 09 and cover the case where the fre¬ 
quency characteristic of the filter <f> 2 is a functional of the signals 
Sxi ( t ). These equations lead readily to a number of necessary con¬ 
ditions for the stability of the self-adjustment process, and also to the 
desirability, with a high-frequency sinusoidal search signal Sxi(t), 
of using two phase discriminators with reference voltages in quadrature, 
so as to make use also of the phase modulation on the carrier signal. 

As a simplified mathematical abstraction, detailed consideration 
is given to the case of an almost periodic signal' 6 (t) and a test signal 
similar in nature to white noise. Some attention is devoted also to quasi¬ 
stationary self-adjustment modes of operation. Illustrative examples 
are given of the calculation of actual systems with several adjustable 
parameters. 

Sommaire 

La communication s’attache a l’ensemble de: (1) deuxfiltres lineaires 
5> 1 et <D a » connects en parallele, recevant un signal d entree 0 (t) et 
(2) des sysiemes cn boucle fermee destines & adapter les parametres 
Xt du filtre <I> 2 a l’aide d’une modulation exploratoire dxi (t) de ces 
parametres, Le circuit auto-adaptatif comprend un detecteur (lineaire 
ou quadratique) de la grandeur de sortie du filtre (D 2 ”* des discri- 
minateurs de phase et des filtres de moyenne W f p. 

A partir de Fhypothese que les signaux dxt (t) sont petits, on 
etablit des expressions approch^es mais qui, dans les limites de la 
m6thode du petit parametre, sont parfaitement exactes, determinant 
la grandeur des actions de r6glage dans les circuits d auto-adaptation 
en fonction des caracleristiques de frequence des filtres <J> X et ® 2 
et des spectres des signaux 3xt (t) et 6 (t). Les equations differen- 
tielles obtenues pour les processus auto-adaptatifs tiennent compte 
du fait que la capacity de memorisation et la bande passante des 
filtres et <D 2 et W (I > sont limit6es; elles s’appliquent dans le cas ou 
la caracteristique de frequence du filtre <& 2 est une fonctionnelle des 
signaux dxt (t). Ces equations conduisent directement k un certain 
nombre de conditions ndcessaires pour que le systeme auto-adaptatif 
soit stable. Elles mettent en evidence Finteret qu’il y aurait, avec un 
signal exploratoire &xi (0 sinusoidal a haute frequence, a recounr a 
des discriminateurs de phase biphases, a tensions de reference en 
quadrature, ce qui permettrait d’utiliser egalement la modulation de 

phase de la frequence porteuse. , .. 

A titre d’abstraction mathematique simplifiee, on etudie en detail 
le cas ou le signal 6(t) est presque periodique et ou le signal explora¬ 


toire a la constitution d’un bruit blanc. On s’attache aussi quelque 
peu aux modes d’auto-adaptation quasi-stationnaires. Des exemples 
illustrent le calcul de systemes existants avec plusieurs parametres 
reglables. 

Zusammenfassung 

Der Aufsatz betrachtet ein Regelsystem das 1. aus zwei parallelen 
linearen Filtern und <D 2 mit einem Eingangssignal d (r)und 2. einem 
geschlossenen Kreis zur Einstellung der Parameter X% des Filters <I> 2 
besteht, welche fiir die Selbsteinstellung eine Modulation dxt ( t ) dieser 
Parameter verwendet. Das selbsteinstellende Netzwerk enthalt einen 
linearen Oder quadratischen Detektor fiir das Signal am Filterausgang 
(<E> 2 — $j), Phasendiskriminatoren und Glattungsfilter W 0 . 

Bqi der Annahme, daB die Amplituden der Signale Sxi (t) klein 
sind, findet man fiir den Regelvorgang der selbsteinstellenden Netz- 
werke Naherungsausdrucke, die den Frequenzgang der Filter und 
<F 2 und die vorliegenden Spektren der Signale dxt ( t ) und 6 (t) ent- 
halten (innerhalb der Grenzen der Methode der kleinen Parameter 
sind diese Ausdrucke vollig exakt). Die fur die Selbsteinstellung abge- 
leiteten Differentialgleichungen beriicksichtigen die begrenzte Speicher- 
fahigkeit und die Bandbreite der Filter <&!, €>2 und W& und gelten 
auch fiir den Fall, daB der Frequenzgang des Filters © a eine Funktion 
des Signales dxi (t) ist. Diese Gleichungen fiihren bereits zueiner Anzahl 
notwendiger Bedingungen fiir die Stabilitat der Selbsteinstellung. Sie 
zeigen auch, daB bei sinusformigen hochfrequenten Signalen zur 
Selbsteinstellung dxi (t) die Verwendung zweier Phasendiskriminatoren 
mit quadratischer Richtkennlinie wiinschenswert ist, um auch die 
Phasenmodulation des Tragersignals auszunutzen. 

Zur vereinfachten mathematischen Darstellung wird der Fall eines 
fastperiodischen Signals 6 (t) und eines Testsignales, das weiBem 
Rauschen ahnelt, genauer betrachtet. Ein weiterer Abschnitt befaBt 
sich mit der Selbsteinstellung bei quasi-stationaren Vorgangen. Die 
Arbeit enthalt durchgerechnete Beispiele ausgefiihrter Systeme mit 
mehreren einstellbaren Parametern. 


Introduction 

§ 1. This paper considers an automatic system (hereafter called 
System A) that consists of linear continuous filters and <E> 2 
connected in parallel, with a test signal 6 (f) at the input and 
closed-loop astatic systems for adjusting the parameters 
X - (*i,..., X N ) of filter <D 2 (see Figure 1). The self-adjusting 
circuit includes a detector 2 of the error signal a (4 phase 
discriminators <E>^, averaging filters W ¥ and integrating net¬ 
works. The control actions in the parameter-adjusting circuits 
are formed by using a search modulation fi Ax (f) of the para¬ 
meters. The defined parameters Y 1 and Y 2 of filters <!>! and <D 2 
respectively vary with time according to a law that is only 
known approximately beforehand. 

In practice, the following variants of System A are most 

often met: 
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Figure 1 


(1) Y x = const., Y 2 = Y 2 (t). The filter <D X is a stationary 
calibration display unit, while the filter <D 2 is an automatic 
system with extremal adjustment of its correcting elements, 
compensating to a given extent the drift of the parameters 
Y 2 (t) 1 - 4 or the variation in the form of the external action 
0 (/)«. 

(2) Y 1 = Y x (t\ Y 2 = const. Filter is a controlled plant 
with variable dynamic properties, while filter ® 2 is a learning 
model of this plant 6 . 

Of course, the general case Y x = Y x (f), Y 2 = Y 2 (/) is also 
possible in practice; for example, a calibration display unit <[> x 
with programmed parameter variation. 

§ 2. In Part I of the paper the small-parameter method is 
used in deriving enough general approximate equations for the 
processes of self-adjustment in System A under the assumption 
that the amplitudes of the search signals juAx (t) are small. The 
equations take account of the limited memory of filters <S> X and 
<X> 2 , and cover the case of any given explicit test and search 
actions. The control signals in the self-adjusting circuits are 
expressed in terms of the frequency characteristics of filters ® x 
and <I> 2 and the spectra of signals 0 (t) and /uAx (t). In Part II 
the general equations of motion for System A are simplified, 
taking the assumption that the search signals fiAx(t) are 
sinusoidal. Then, as a simplified mathematical abstraction, the 
case of an almost periodic action 6 ( t ) is examined in detail in 
Part III. A very simple analysis of the relevant equations of 
motion shows the desirability, with a high-frequency sinusoidal 
signal fxAxi (/), of using, in the phase discriminator, a reference 
voltage phase shifted with respect to this signal, which permits 
one to make use of the extra useful information carried by the 
quadrature component of the search-frequency signal, by anal¬ 
ogy with the practice, in radio engineering, of using amplitude 
and phase modulation simultaneously 7 . Part IV uses the example 
of a white-noise test signal to show that the equations derived 
may also be applied to the description of System A with stochas¬ 
tically defined signals 0 (f), without relying on the hypothesis of 
the closeness of random processes in the system to stationary 
ergodic ones. There is a brief discussion of the relation between 
the results derived here and those in previous papers 1-6 . Some 
attention is also devoted to quasi-stationary modes of self- 
adjusting operation. 

In conclusion it should be stressed that all the design examples 


quoted have been chosen to be simple as far as possible, and that 
the main emphasis is on the physical interpretation and quali¬ 
tative analysis of the mathematical relations derived. 


I. Derivation of General Equations of Motion for the 
Self-adjusting System Considered 

§ 3. The most important of the assumptions, under which 
the equations for the processes of self-adjustment in System A 
are derived below, are first set out: 

(a) The amplitudes of the search-modulation signals are 
considered small, and to emphasize this they are denoted by 
/uAx (f) where ju is a small parameter. 

(b) It is assumed that System A starts to operate at a certain 
instant t = t 0 , having been in an equilibrium condition up to 
that time, and thus the output quantity of filter <D e (/ = 1, 2 ) is 
determined by the relation 

ii(t)= 6(x)Ki(t, t)dr, i = l,2 (1) 

J t 0 

where K t (r, r) is the weighting function of filter d> ?: . 

Equation (1) is expressed in the form 

ft-r ft 

£i (0 = 0 (t) Ki ( t , t) dr + 6 (r) K t (/, x) di, i = 1,2 

J to Jt-T 

t 0 <t—T <t (2) 

Let the filters <£>*• be stable. Then if 


|0(OI<const (— oo<t< + oo) (3) 


it may be considered that for a certain sufficiently large T the 
first integral in eqn ( 2 ) is negligibly small, and 


m- 


6 (t) K t (t, x) dt, i = l,2 


(4) 


In other words, this means that by the instant t information on 
the state of filters O,- and on the values of the signal 6 (r) at 
instants r > t — T is practically completely lost, and the value 
of ^ (t) may be identified with the reaction of filter to the 
signal 

e <y) = f 0 ( T ) for 

T (0 outside that interval ( 5 ) 

assuming that (t) = 0 for r < t — T. 

The conditional nature of any choice of a numerical value 
for T matches the complexity of the actual situation: the 
‘memory’ T of a linear system depends substantially on the 
criterion chosen and on many factors that are often not subject 
to any sort of accurate quantitative calculation (on the structure 
of the signal 6 (t) within the bounds of the natural and easily 
enough controlled restriction (3), on the level of fluctuating 
disturbance in the system, etc.). If filter <D Z * is near to the stability 
boundary in parameter space, then of course oo. If stability 
is lost then (4) is not true even for T = oo, and strictly speaking, 
one cannot apply either the theory developed below, which takes 
no account of the initial perturbations always existing in a 
system, or the theories of Krasovski 2 , Kazakov 3 and Varygin 4 . 

(c) It is also considered that the variation in parameters 
Y x (t), Y 2 (t) and X (t) over the time interval Tmay be neglected. 
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The time-dependence of the frequency characteristics W 1 (jco) = 
Wj(jco, t) and W 2 (jco) = W 2 (jco , /) of filters €>* and 0 > 2 (with 
Ax (t) = 0 ) is only expressed in the taking of the values of 
parameters Y x (t), Y 2 (t) and X ( t ) as ‘frozen’ at the given 
instant t: 

Yi (t)- Yt ( t), Y 2 (t)= Y 2 (t), X (r)=*X (t) for t—T<r<T 

( 6 ) 

(d) Finally, for the sake of definition, it is assumed that the 
state of filter 4> 2 is described by the ordinary differential equation 

n m 

la 2 , k (Y 2 ,X+ M Ax)D k £ I (0=5Xi(r2,X + Mx)D' 9(f) 

k = 0 1=0 (n\ 

n d w 

D = d ? m ~ n 


liAx t (jco) = n i Ax f (z)-e }<ax dr (13) 

J — 00 

of the search signal. 

§ 5. A solution i 2 (A to eqn (?) is looked for in the form of 
a series 

5 2 (0-«2o(0+^2i(0+... ( 14 > 

all the analysis below being taken only with the accuracy of 
magnitudes of the first order of magnitude with respect to the 
quantity p [obviously one way of making the theory more 
accurate is to take account of more terms in (14)]. Using the 
normal procedure 9 for the small-parameter method, the follow¬ 
ing equation for sequential calculation of the quantities £ 2 o (0 
and i 21 (t) are obtained from (7)-(9): 


with coefficients a 2)]c (Y 2 , X) and b 2 , l (Y 2 ,X) that are analytic 
in X. It is evident that 

W 2 (jco )=Ri (jco) • Qz 1 (jco) ( 8 ) 

where 

Qz(D)=t a 2 tk (Y 2 ,X)D k 

k = 0 

( 9 ) 

R 2 (£>)=Z b 2A {Y 2 ,X)D l 

1 = 0 

Given assumptions (a), (b) and ( c ) the proposed method of 
calculation can be generalized without much complication to 
the case where pure delays are present in the filter <£> 2 under 
adjustment. 

§ 4. It is observed that assumption (b) allows one to make 
calculation in a frequency region bounded only by consideration 
of the ‘shortened’ present spectrum 8 

0 (T)e"^ r dT= ' d T (r)t~ J<ox dr ( 10 ) 
J t-T J - oo 

of the signal 0 ( t ). Thus, in particular, taking into account the 
quasi-stationary nature of the filter W x (jco, t) the following 
relations are obtained for ( t): 

£i(0=J- f” ( 11 ) 

-oo 

where 

{1 (jco, 0 - w t (jco, t ) • e T (jco, t) ( 12 ) 

Considering, instead of normal spectra, the ‘shortened’ 
present spectra of the type in ( 10 ) and ( 12 ), generally one can 
reflect more accurately in a mathematical model the actual 
situations that arise in the experimental development of System A, 
and also simplify mathematical operations on the spectra of the 
signals 6 (t) and Si (0 in those cases where the Fourier integrals 
for these functions over the interval (— go, t) diverge. This 
approach turns out, in particular, to be very convenient for the 
examination of non-ergodic random processes in System A, as 
it gives a natural transition to the description of the system in 
terms of spectral power densities (see Part IV). 

Since this paper only considers explicit (and, what is more, 
only harmonic) search signals pAx (t), from now on in order to 
simplify the text the ‘full’ spectrum is used as a convenient, if 
less accurate, mathematical abstraction 


Q2(D)'Z2o(t) = R 2 (D) 6 (i), Ds- 


Q 2 (m. 


,l(0=£ AX;(0 ' 

i = 1 




It is easily seen that given assumption ( b ) the memory of the 
linear system (16) should be considered as limited to the time 
interval T. Hence, taking into account the quasi-stationary nature 
of the filters Wi (jco, t ) and W 2 (jco, t) and the identity d WjbXi 
= 0 , the following expression is found for the ‘shortened’ 
present spectrum of the error 

£ (t) = fao (0 - #i « " « ( 17 ) 


e (jco, t)—W (jco) 0 T (jco, t) 
u i ,. * f 00 dW(jv) 

+ r« e > UW> ,SJ ..~wr 


Qi(j v ) 0 

Ax £ (j(a> —v))*dv (18) 


where W (jco = W 2 (jco) — W 1 (jco) 

while the spectra 0 T (jco, t) and A Xi(jco) are defined by eqns ( 10 ) 
and (13). 

Furthermore, in accordance with the circuit shown in 
Figure 1, one obtains for the X r adjusting network 


DX^W^cpisit^Ax^t), D~ 


where (p (e) is the detector characteristic, while 

8 (t)=U f £ (jco, t) ■ e Jat dco ( 20 ) 

2nJ -oo 

The approximate system of eqns (17)-(20) that has been 
obtained describes a very wide class of self-adjusting operating 
conditions for System A. 

The following points are stressed: 

(1) These equations, written in terms of the frequency 
characteristics, are differential equations (generally speaking, 
non-linear) and in a number of cases are capable of more 
effective investigation than the integro-differential equations 
derived by Krasovski 2 and Varygin 4 in terms of weighting 
functions. 
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(2) In distinction to the previous papers quoted 2 ' 4 , the 
derivation of eqns (17)-(20) does not rely on the assumption 
that the weighting function, and consequently also the transfer 
function, of filter €> 2 is actually a function rather than a func¬ 
tional of the signals juAx (t). 


II. Simplification of the General Self-adjustment Equations for the 
Case of a Harmonic Search Modulation and a Square-law 
Detector 

§ 6 . If during the whole time of operation of System A the 
search jlcAx ( t ) is nearly harmonic, then it is convenient to 
consider that 


Z i — 1 

1 N 

cos((at+(p+a)+—n £ A;B;( co,t) 

Z i= 1 

sin(co? + (p + a)||0 r O'ffl,O|dco (27) 

where 

A,(co, t)=a i (co)cos Q,t + b i (a >)sin Qpt 
a ‘ (") cos <P? (“) + M i~ (a>) cos (co)) 


s(t)—n 1 


Ax i (t)=A i cosQ i t for -co<t<co, i = l,...,N (21) 

(£2i < Q i+ j) 

Then in accordance with (13) 

Ax i (jco) = n[S(co + i2 i ) + S(co-Q i )] . (22) 

Substituting (22) in (17) and using the known properties of 
8 functions, one finds: 

s (jco, t)=W ( jco ) 0 T (jco, t) 

+ y^ 22 ~ 1 0 '®) x 

.E A i q 2 {j(o>+ 12,.)} e T {j(co+Q t ), t} 


~~ I W (yco)| g~(M 2 + (co) sin <p £ + (co) + M t (co)sin^ (co)) 


b. (ft)) = ^g^(Mf (co) sin ( p~ (co) - M ; + (co) sin <p; + («)) 
-|lf ( 7 'ft))|^(Mj + (ct))cos^ ; + (co) — Mf (co)cos cp(~ (co)) 


(28) 


QzUa) 


o 2 {j(oj-Q:)} 
0.2 (JCO) 

Q 2 {j(a>+Q i )}' 


J(pr O) 


=M; - (co)e J> ‘ 
=M i + (co)e M+( " ) 


(29) 


+ ~~^ 8 Z i { j ((0 - 12 ;)} 0 T { j (ft) - 12 ;), t} 

Then let 


(23) 


£( 0 = 


1 


|lF(jco)| cos (coH-<p + a) 


+ y'/t X A; [2?eC; cos (cot + a) 


i=l 


— ImC; sin (cot+a)] | 0 T (jco, f)| J- dco 


r a^Qo) 

C ) = —gy—<2 2 0®) 


-/Oif 


-j&it 


.02 {j((0- 12;)} + e 2 {; (co +12;)}J (26) 

In calculating the passage of the signal e ( t ) through the 
detector it is convenient first to separate, in each term of the 
integrand in (24) that is enclosed in square brackets, the com¬ 
ponents in phase with the signal cos (cot + cp + <x) and those 
in quadrature with it. One obtains as, a result: 


[the expressions defining the coefficients B t ( co , /) are analogous 
to formulae (28) and (29)]. In quasi-stationary self-adjustment 
modes, when is small compared with the actual frequencies co 
of the test signal 6 (t), Q 2 (jco) =£= Q 2 (/(co ± 12*)), so that 


W(j(o) = \W(jco)\ Q J(p (cp^cp(co)), 

&T (jco, t) — \9 t {jco, t)\ e ya (a = a (co)) (24) 

Taking into account the even nature of the amplitude spectrum 
and the odd nature of the phase spectrum in (24), one can 
readily deduce from ( 20 ) and ( 22 ) the following expression for 
the error e (t ): 


a ,•(<»)—2 


Mf Q J<Pi ~=1 —M* e J<Pl + 
Q\W(ja>)\ 


dX t 


(30) 

bi(co)^ 0 (31) 


§ 7. Most often the detector ^ may be considered as either 
square-law: 

?(e) = £ 2 (32) 

or linear: 

cp(e) = |e| = (6 2 )* (33) 

in both cases the theoretical analysis requires the square of the 
error e (t) to be calculated. Taking only terms of zero- and 
first-order magnitude with respect to quantity /c, the following 
expression is readily derived from (27): 


(25) 

s 2 (0—: 

(26) 

where 


1 _ 

2tc 2 


D {co , v) [cos {(co - v) t + - S v ) 


+ cos ((co + v) t + 3^ -f- 3 V )] dco dv 
D (co, v)^=\e T (jco, t) I |0 r (jv, 011 W(jCD) 11 m,/v)l 


(34) 


X 1+y J] A; {A; (ft), f) | fk ( jCO)\ 1 
Z 1=1 


+ X;(V,0|^(jv)|- 1 } 


(35) 
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While 9 a = q>(a>) + ot(a>) + ii/(a>) (36) 

tan i^(ai)=-y £ A ; B;(co,0 \W{joS)\+~ Z A^co.t) 

2 i = 1 L 2 J=1 

In all the following, harmonic search-modulation signals 
are, in fact, considered and as a mathematical model of System 
A one takes the system of eqns (19), (32)—(36). These equations 
continue to hold adequately until the instant when through 
the operation of the self-adjustment circuits the relation 

I W (jco)|=/iJV max A ; \A i (co, *)| (37) 

i, t 

becomes true (in this event the approximate expressions in (35) 
for 3) (a>, v) are already invalid). 


III. Theoretical Analysis of Self-adjustment Modes with an 
Almost Periodic Test Signal 


§ 8 . If over the whole time interval that this paper is con¬ 
cerned with the test signal may be represented accurately 
enough in the form 

M 

0(t)=0 o +Y J O k cos(a> k t+a k ) 

fc= 1 

9 k ,a k ,(o k = const, co k <co k+1 (38) 

then it is convenient to consider (38) as being true for 
— oo < t < -f oo. Then 

M 

\d r (jco,t)\=n Z 0 k d(co-co k ), co 0 = 0 (39) 

k — 0 


and in accordance with (32)-(35) 

M 

e\t)=- Z O k O l \WUco k )\\W(M)\e kl (40) 

Z k,l — 0 


= jJ+yZ o | W(jco k )\ 1 +A i (co l , t) | W (jco t )\ 1 

x [cos ((to* - (o,) t + $ k - 9,) + cos ((co* + mi) t+0 k + 9,)] (41) 


where # 7c and A t ( co k ,1) are as defined by (36) and (28). 

It can be seen from (41) and (28) that the signal e kl is made 
up of a sum of harmonics at frequencies oo k ± co h oo k ± co x + Q s 
(k, l — 0,..., Af, s — 1,..., N). In the phase discriminator of the 
zth self-adjustment channel the output quantity cp ( s) of the 
detector 3) is multiplied by the harmonic reference voltage at 
frequency £2 i9 so that with square-law detection a signal (t) 
is obtained consisting of harmonics at frequencies co k ± (Oi± Qi, 
Qj k ± co l + Q s ± (with linear detection in general one also 
gets other harmonic components with amplitudes that are first- 
order of magnitude with respect to quantity ju). 

If in (38) 6 0 represents a slowly varying useful signal, while 
the sum M 

Z Ok cos (m k t + a k ) (42) 

/{= 1 

represents intense disturbances at sufficiently high frequencies 
(&>! > 2 Qjy), then correctly chosen smoothing filters 
W$i (D ) should pass only harmonics of the signal U{ (t) with 
frequencies 

(o k -coi-(Q a ±Q3> ( 43 ) 


It is assumed that the disturbances (42) acting on the system 
are such that for k > l the conditions 


co k co^Qi, co k -co l ¥=(O s ±O i ) (44) 

are satisfied with adequate margin. Then it may be considered 
that the constant component in the signal (t) that is passed 
by the filter (D ) accurately matches that harmonic of the 
sequence oo k — co l — ( Q s — Qj) for which k = l and s — /. 
Then according to (41) and the circuit of System A 


where 


w 9i (D) • E t (s 2 (t) m ic cos t ) 

E, [/ (t)] = lim 2- I* / (t) dr 


(45) 


and the values of the parameters X , Y ± and T 2 in the expres¬ 
sion for e 2 (t) are taken as ‘frozen’ at the instant t [see (6)], so 
that 

E, [c 2 (t) m ic cos Q i l']=k fl A i m ic + E (2) + £'.?>] (46) 


o M 

O 2 1 W (jco k )\ 2 (M f t cos (pf k +Mf k cos cpf k ) (47) 

OA jfc = 0 

M ^ 

E ( C V=- Z 0\ I W(joi k )\ 2 ~~(Mtk cos (ptk + M ik cos cp ik ) (48) 

k = 0 AO i 


M 

EiV=~Y 0 2 k \W(jw k )\ 2 


k = 0 


9 <P(ft4) 

dXi 


(M^ sin (pf k +M ik sin (p ik ) 
(49) 


where the quantities M^ c eJvt k are defined by formulae (29) 
with co — oo k . 


§ 9. The case of the quasi-stationary mode of operation 
(0 o = 0 and condition (30) satisfied) are first considered. 
Equation (45) for the self-adjustment process becomes 


a y i c) N 

(D )' ~2 E\ m ic I el -\W(jco k )\ 2 (50) 

and thus as a result of the normal operation of the self-adjusting 
circuit (without loss of stability, without intense distortion 
caused by disturbances etc.) the quantity 


M 

Z Ol\W 2 (joi l f-W x (joi k )\ 2 (51) 

k= 1 

will be a minimum, i.e. in the complex plane the frequency 
characteristic of the filter being adjusted will approach that of 
the calibration filter at the points oo = oo h (/: = !,..., M) in 
some mean-square sense. If by varying the adjusted parameters 
X the frequency characteristics W 1 (joo) and W 2 (jco) can be 
made practically identical over some range of frequencies, then 
this approach will merely signify that over the given frequency 
range W x (jco) == W 2 (jco), and the result of the normal operation 
of the self-adjusting circuit will prove practically independent 

' M 

of the actual spectral composition of the test signal E 0 ]c cos 

/c-l 

(co k t H- & & ), (coj^ > Qjsr) (see Example 1). The latter statement 
is not valid (see Taylor 5 , and also Example 2) if the filters 
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Wj (jco) and W 2 (jco) essentially cannot be made identical. 
In this event the closest convergence of the frequency character¬ 
istics W 1 (jco) and W 2 (jco) takes place at those points co — co k 
corresponding to large amplitutes 0 k , and the nature of this 
convergence will change with variation both of the frequencies 
co k and of the ratios between the amplitudes 6 k . 

Example 1. In a System A with square-law detector, let 
filter W 1 be a controlled plant with transfer function W 1 (p) = 
(b%p + bi)” 1 , and filter W 2 a self-learning model 6 with a transfer 
function of the form W 2 (p) — (X 2 p + X^~\ where X x and X 2 
are the adjustable parameters, by varying which a complete 
identity between the dynamic properties of model and plant 
can, in principle, be achieved. If 

0(t) = 0cos(£t4-a), Q$>Q 2 >Q 1 (52) 

then eqn (50) for the quasi-stationary self-adjustment mode 
takes the following form: 

X^k.W^iD) 

■ [(.Z! - b x ) (Zf Q 2 +X 1 b 1 )-Q 2 (X 2 - b 2 ) 2 ZJ (53) 

X 2 =k 2 W, 2 (D)Q 2 

■ [(Z 2 - b 2 ) (X 2 +X 2 b 2 Q 2 )-X 2 {b,- x ,) 2 ] (54) 

k i =^ i mi c e 2 (b\ + blQ 2 )- 1 -{Xl + X 2 2 Q 2 )- 2 , i= 1,2 (55) 

It can be seen fromeqns (53)-(55) that as a result of the nor¬ 
mal operation of the self-adjusting circuit X 1 -> b x and X 2 -> b 2 , 
i.e. in fact W 2 (jco) -> W x (jco), at whatever frequency Q the test 
signal (52) is applied. The self-adjustment process forms a 
coupled control of the parameters X x and X 2 . The higher the 
frequency Q of the test signal, the more intensively the adjust¬ 
ment of X 2 takes place (cf. Margolis and Leondes 6 ). The stability 
for small variations of the equilibrium X x — b l9 X 2 = b 2 in 
the non-linear system (53)-(55) can readily be examined from 
the first-order approximation equations. 

Example 2. If in the system just considered the controlled 
plant is close in its dynamic properties to the link W x (p) — 
e~ pr (V + bi)~\ while W 2 (p) = (X^p + X x Y\ then for r ^ 0 
complete identity of filters W 2 and W x cannot be achieved by 
self-adjustment. Transcribing eqn (50) for this process, it can 
easily be seen that the result 

X 1 -+b 1 cos Qz — b 2 Q sin Qz , 

. X 2 -*b 2 cos Qz + b^" 1 sin Qz 

of normal operation of the self-adjustment network may 
already depend substantially on the frequency of the test 
signal (52). 

§ 10. Eqns (45)-(49) also permit a number of conclusions of 
a qualitative nature on non-quasi-stationary modes of self- ad¬ 
justment in system A (0 O # 0, conditions (30) not satisfied) to 
be immediately drawn. 

It is first observed that the equation 

62 = 620) ( 56 ) 

defines a Mikhaylov hodograph 10 for a stable [assumption (6)] 


linear system, and consequently the curve (56) has a form 
similar to that in Figure 2. 

It then becomes clear from (45)-(49) that within the limits 
of the errors introduced by the terms E { $ and E% ] the normal 
operation of the zth self-adjustment channel reduces to the 
conditional minimization of the quantities 

M 

Z e k \W (M)I 2 (Mg cos cptk + Mg cos <pg) ( 57 ) 

k= 0 



The self-adjustment error associated with j E®1 will be small 

in most cases, since by the very sense of the quantities Mf k eMfc 
[see (29) and also Figure 2] the partial derivatives bjbXt (Mf k 
cos q>M. + cos (p m) will hardly be significantly different 

from zero. The error associated with E { $ will also be insignificant, 
since with \cpf k |, \cp^\ < n the terms M? k sin(pt k and 
sin cp m are opposite in sign. 

If 

Mtk cos cpg +Mg cos cpg > 0, k=0,...,M (58) 

then the conditional minimization of the quantities (57) has rough¬ 
ly the same physical significance (see § 9) as the minimization of 
quantity (51) in quasi-stationary modes, and thus the result of 
normal operation of the self-adjustment network should be taken 
as acceptable. But the more strongly the self-adjustment mode 
differs from quasi-stationary, the larger are the angles \cp%\ for 
\cpf k \< n the smaller are the coefficients in (58) (inparticular, 
the quantity cos cp ^ 4- cos cp ^), since over a sub¬ 

stantial range of frequencies the quantities M-j c are hardly 
much different from unity. As a result the quality factor for the 
Xi tracking system falls, while for \(pf k \ > tz/2 the coefficient 
(58) becomes negative and minimization of the weighted sums 
(57) of squares | W 2 (jco k ) — W x (, jco k ) | 2 loses its evident sense, or 
even an inversion of the self-adjusting servo-system occurs 
(particularly if all the coefficients (58) become negative, which 
may happen if a strong signal 6 (t) is applied at a frequency 
close to a search frequency Q i —see Example 3). 

Example 3. Let the adjustable filter in System A be a link 
with transfer function W 2 (p) = kQ^Y 1 = k(p % + 2 ocp + co^Y 1 , 
the adjustment parameter being the gain k, modulated by a 
signal A k * cos Qt, while to the input of the system is applied 
the test action d (t) = 0 cos (cot -f oc). 
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Writing out the general expressions for the quantities 

M ± Q j(p + = Q 2 ( jco ) • Q 2 [j (co ± Q)J - 1 (59) 

it can readily be established that condition (58) for the system 
considered is explicitly unobserved if co Q is small (c%->0), while 
the frequencies Q and co of the search and test signals coincide 
and exceed co 0 , since then 

M + cos q> + + 2 a 2 ) ( Q 2 + « 2 )~ 1 

M “ cos cp~->— Q 2 cdq 

M + sin cp + -> - -i- a (Q 2 + a 2 ) “ 1 

M~ sin cp~ ~->2ccc0q 2 

and the coefficient M~ cos cp~~ becomes in (58) greater in modulus 
than the coefficient M+ cos cp + . 

It is observed that since in this case the quantities (59) are 
independent of k, the error associated with the term E { H in 
eqns (45)-(49) proves equal to zero [this situation will occur 
every time that the adjustable parameters of filter 0 2 appear 
only in the numerator of the transfer function W 2 (p)]. 

Considering eqns (40), (41) and (28), it is noted that to 
increase the capability of the self-adjusting circuit for operating 
in non-quasi-stationary conditions one may use in the phase 
discriminators <D D t the reference signals 

m ic cos Q(t + m is sin Q t t (60) 

which are phase shifted with respect to the search modulation 
signal 

pA^os Q t t (61) 

In this case the processes of self-adjustment will proceed in 
accordance with the equations 

^i— w < P i(D) [E, (s 2 (t) m ic cos Q t t)+E, (s 2 (?) m is sin (<)] 

(62) 

where E t (s 2 (t ) m i& cos Q t t) is determined by formulae (46)-(49), 
while 

E t 0 2 (0 m is sin = j-M, m, v (£<’ > •+ E< 2) + E ( s ?) (63) 

0 M 

Es ‘ )= W £ °k \w (jco k )\ 2 (M ik sin (p ik -Mi k sin <pf k ) (64) 
OAik^o 

M 0 

E[ 2) =-'Z 0 2 k \W (jco k ) I 2 — (M ik sin <p ik - M? k sin <p? k ) (65) 

k- 0 

eT= - E 0 2 \W(jcD k )\ 2 ^^(M+coscp: k -Mr k cos<p7 k ) 

,,=0 SX( ' ( 66 ) 

Here the necessary condition (58) for normal operation of 
the self-adjusting circuits is replaced by the condition 

M tk (m* cos (ptk ~ m is sin <p? k ) 

+M[ k (m ic cos (pi k +m is sin <p^) > 0 (67) 


which may prove much more favourable given a suitable choice 
of the phase of the voltage (60) (i.e. of the quantities m ic and 
m is ); the actual result of the undistorted forced process of self¬ 
adjustment comes out in this case to be the conditional minimi¬ 
zation of the quantities 

M 

E Ql\W (M) i 2 [M* k (m ic cos q>f k - m is sin q>? k ) 

k = 0 

+ M ~k (Mic cos cpr k + m is sin <p~ k ) (68) 

In choosing the phase of the reference voltage (60) one can 
aim not only at increasing the coefficient (67) but also at the 
same time decreasing the quantity 

|M ; t (m ic sin <p+ k + m is cos <pf k ) 

+ M 7k (™ ic sin <pr k + m is cos <pf k )\ (69) 

i.e. (see (62), (66) and (49)) the error associated with the term 
E { H. In practice, as a rule, it proves tedious to achieve an 
accurately optimum phase-shift (e.g. in the sense of a minimum 
ratio between the quantities (69) and (67)) between the signals 
(60) and (61), since by virtue of (29) this shift depends not only 
on the drifting parameters of filter <fi 2 (a similar situation 
arises 11 also in extremal control systems), but also on the form 
of the test signal 0 (t). Nevertheless by using a priori information 
on the operating conditions of the system, or by carrying out 
a running analysis of the signal 6 ( t ) and the results of system 
operation, in a number of cases one can evidently achieve an 
improvement in the dynamic properties of the given self-adjusting 
system relatively simply by using reference voltages of the form 
in (60) that only approximate to the optimum. In order to 
increase the stability of automatic phase-shift optimization 
between voltages (60) and (61) one can correlate the search and 
test signals in frequency [phase relations between the signals 
6 (/) and cos Q t t have no effect on the quantities (47)~(49), 
(64)—(66)]. 

The self-adjusting system, the phase of which use discrimina¬ 
tors reference voltages of the general type given in (60) will 
be denoted by System B. 

§ 11. The equations of motion (45)-(49) and (62)-(66) were 
derived under the assumption that the frequencies of the search 
modulation and the harmonic components of the test signal all 
satisfy the conditions (44). If these conditions do not hold, then 
the voltages E t (e 2 (t) m ic cos Q { t) and E t (e 2 (/) m is sinl^r), 
together with the signals (47)-(49) and (64)-(66), will also 
contain other components, which generally speaking will 
introduce certain additional distortions into the self-adjustment 
process. Equations (40) and (41) enable one effectively to 
calculate all these parasitic components of the control signal in 
the self-adjusting network. 

For example, let only one of the conditions (44) be disturbed: 
let the frequency of the pth harmonic of the test signal coincide 
with the search frequency in the zth self-adjusting channel, 
i.e. co p = Q t , According to (41), in this case the signal 
E t (s 2 (t)m ic cos Qit) will contain an additional term E { % 
generated by the presence in e kl of harmonics with frequencies 
CO), j. -f coi — Q q (for k = 0, l = p t q = i and k — p, / = 0, q = i) 
and co k + — (Q s + Q a ) (for k = l = p, s = q = i): 
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E ( cV =y m, e E, [I %e p \W (0) W(jco p )\ (e po + e op ) 

+Y 0 2 P I W (jcop) | 2 e pp ] cos Q, t 
= rn ic e o 9 p \W(0)W(jco p )\cosS p 

+jMi m ic &l | If 0'co p )| x [a, (co p ) cos 2 9 p 

- b t (co p ) sin 2 9 p ] (70) 

where = # (cc^), a* (co^) and b t (co#) are defined respectively 
by eqns (36), (28) and (29) with co= cap. 

For the system considered in Example 3, the first term in 
expression (10) is zero (since 0 o — 0), while the second may be 
calculated given the frequency characteristic of W 1 (jrco). Even 
in this actual example it is, on the whole, difficult to judge what 
effect the use of a reference voltage of (60) type will have on the 
additional error in question. One can evidently achieve a stable 
reduction in this error or even its conversion into a useful 
signal, provided one correlates the search and test signals not 
only in frequency but also in phase, so as to limit unforeseen 
variations in the angle 


IV. Calculation of Self-adjustment Operating Modes where the 
Test Action is a Stationary Random Process 


while the integral 


is a weighted sum 


i: 


G q (co, v, t) dr 

1 t-T 0 

of integrals of the type 


< 

J t-T 0 


cos (Qr-{-9) dr 


(76) 


where Q are the frequencies in (75) and $ are angles of the form 
#g> ± # v and ± # v + njl. On rewriting the integral (76) 
in the form 



/ j \ 

cos 

QiZ+t-^T, +9 

J -zr 0 

L V 2 / J 


it is observed that in accordance with a known 8 integral repre¬ 
sentation 

1 f 00 

Y n 1 cos(12T + 9)dT = cos9-<5(12) (77) 

^ J - oo 


of the 3 function and for large enough averaging time intervals T 0 
of the filter W$ q (Z>), the approximate equation 



cos (Qz -f 3) dr =a= cos 



= cos 9 • 5 (Q) (78) 

is true; using this, eqn (74) can be readily got into the form 


§ 12. It is assumed for simplicity that the filters JV^(D) in Sys¬ 
tem A consist of elements which carry out the ideal averaging of 
the quantity m ic s 2 (t) cos Q t t in time over the interval (t- T 0 , t): 

W <pi (D)[e 2 (t)m ic cosQ i t']=l^- ( s 2 (r) cos 12 ; t dx (71) 

1 o J t-T 0 

and that the test signal 0 (t) is a time-function whose ‘shortened’ 
spectrum (10) actually only slightly depends on the instant of ob¬ 
servation t and is located in the region of quite high frequencies: 


9-r (jco, t)=z=0 T ( jco ), 9 r ( jco ) = 0 for co < co* (72) 

cq*-2Q n >Tz\ Qi-Q-^To 1 (73) 

(Qi>Q t _ i, i = l,..., N) 


Every actual filter W (jco) = W x (Jco) - IV 2 (jco) has a finite 
cut-off frequency co rp (it is further considered that co* < co ), 
so that in accordance with (19), (32), (34) and (71)—(73) the 
equations for the process of self-adjustment of the #th para¬ 
meter may be put into the form 


dm 


dvr n " 


G t (co, v, t) dr (74) 


G q (co, v, t)= D (co, v) [cos {(co - v) t+ 9 ra - 9 V } 

+ COS {(CO + v)t +9^ + 9,,}] COS 12,T 

where D (co, v), and # v are defined by eqns (35), (36), (28) 
and (29). The quantity G t (co, v, t) is a sum of harmonic com¬ 
ponents with frequencies Q equal to 

g>±v± 12 4 , co + v±((2 s ±£2 ? ), (s = l, N) (75) 


r 

X v ^T-T 0 ~ 1 -k q -n~ 1 G q (co) 

J co* 

a N 

+-J .Z A i ISi (to, vj) + g, (co, v iq ) + g, (co, V M )] dco (79) 

where 

G q (to) = \e T (jco) e T {j(co + 12,)} W(jco) W{j(co + 12,)] | 

•T cos(9 a —9 a i+Q q ) (80) 

g i (co,v)=~ T - 1 \6 T (jco)9 T (jv ) If (jco) W(jv)\ 

' [ Vic (to, v) cos (9 a - 9 V ) - V is (co, v) sin (9„ - 9 V )] (81) 

V ic (co, v) — a i (co)\W (jco) \ ~ 1 + a t (v) | W (jv)\~ 1 
V is (co,v) = bi(co)\W(jco)\~ 1 + b i (v)\W(jv)\~ 1 

v*=co + 12 i + 12,, y~ = co + 112;-12,| (83) 

[the quantities (co), bt (co) and ^ being defined by eqns (28), 
(29) and (36) and T the memory of filter W(joS )—see § 3], 
Considering the function (5) as a typical realization of a station¬ 
ary random process {0(0} and performing averaging according 
to achievements, one can go from eqns (79)-(83) to equations in 
the mean (as taken together) values X q of the adjustable para¬ 
meters. If here the interval T is taken large enough, then in the 
right-hand sides of these equations one may replace the quantities 
7" ,-1 | S T (jco ) 0 T (jv) | by characteristics like the mutual spectral 
power densities 12 of the process {0 (0} and certain random 
processes obtained from {0 (0} by simple transformations that 
do not infringe the stationary condition. 
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This paper does not deal with the more detailed analysis of 
the general case, but gives the results of the calculation for the 
quasi-stationary mode of self-adjustment, i.e. the mode in which 

co*>2Q N (Q i >Q i - 1 , i' = l, N) (84) 

with a test signal of white-noise type: 

T - 1 Wr(M *- Km T ' 1 |0 T (j<o)\ 2 = (85) 


Since eqns (30) and (31) are satisfied in quasi-stationary 
modes, and furthermore ~ ^o>+2Qm ( w > "*)» one ma ^ 
neglect the terms V i8 (co, v) sin (#* - # v ) in (81), and so putting 
0 T (jco)=^6 T {j(a) + 2Q M )} and W(jco)^= W{i(a> + 2Q M )}, the 
following equations for the self-adjustment process are arrived at: 




\W{jco)\ 2 dw + nA q 


dX„ 


\W(ja>)\ 2 da> 


'dX, 


q J 


\W (jco)\ 2 dco 


1 V A 0 

+y/* I A; 

z i = 1 
i*q 

k° — T- Tq 1 ■ k' q • m qc -G 0 


( 86 ) 


1 


The following conclusions are evident from (86): 

(1) In the mode of operation (84), (85) studied, minimization 
of the quantity 

co, p 

\W(jco)\ 2 dco (87) 

J CO* 

may be naturally considered the ideal result of the self-adjust¬ 
ment process. 

(2) The control signal for the cjth self-adjusting network 
contains derivatives of the quantity (87) being minimized, not 
only w.r.t. X q but also w.r.t. all the other adjustable parameters 
Xi , so that one has not got a pure gradient system of extremal 
control. 

(3) The equilibrium condition ~X q = 0 (q = 1, ..., N) for the 
system (86) is characterized for A* = A (/ == 1, .. N) by the 
relations 

r) C °* <p 

M(N+l)- aT | W (joj)\ 2 dco= — \W(jco)\ 2 d(o (88) 

ij co* J a>* 

(/ = 1,..-, AO 

from which it can be seen that the strongest extremal nature of 
the dependence of quantity (87) on the parameters X i9 and the 


less attainable the minimum of this quantity, the closer will this 
condition be to the ideal result of self-adjustment. 

(4) If quite large differences arise rapidly between the 
frequency characteristics W x ( jco ) and PV 2 (Jco), the non-negative 
term (87) on the right-hand side of eqn (86) will increase so 
much that the operation of the self-adjusting network will be 
reduced merely to increasing the parameter X Q (X Q > 0), and 
this may lead to the system’s losing its required extremal 
condition. 

Finally it is noted that the equations given by Krasovski 2 
for quasi-stationary self-adjustment with a white-noise test 
signal contain only terms analogous to the second term in the 
right-hand side of equation (86). 

The author expresses his gratitude to Ye. A. Barbashin and 
L N. Pechorina for their discussion of this paper. 
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Some Considerations on Optimized Integrated Control 

R. MARCACCI 


Summary 

The authors have tried to find an analytical solution to the problem of 
linear integrated continuous control, of the nth order, in an open field, 
by optimizing its operation efficiency with the dynamic programming 
method and according to the point of view of Letov on the problem in 
question. 


Sommaire 

Le rapport se propose de donner une solution analytique au probleme 
d’un systeme de reglage lindaire continu d’ordre n en domaine ouvert 
en optimalisant son fonctionnement par la methode de la program- 
mation dynamique, selon la methode developpee par Letov. 


Zusammenfassung 

In dieser Arbeit wurde der Versuch unternommen, die analytische 
Losung des Problems der gesamten kontinuierlich arbeitenden Rege- 
lung eines linearen vermaschten Systems n-ter Ordnung in einem offe- 
nen Bereich zu finden. Das Optimum der Regelgiite wird mit den 
Methoden des dynamischen Programmierens und nach einem von 
Letov vorgeschlagenen Verfahren gesucht. 


General 

This paper is written with the object of optimizing a continuous 
control process, having a well-known linear differential transfer 
function, depending on several control variables defined in an 
open field according to the dynamic programming method 
applied by Bellman 1 and Letov 2 . 

Suppose that a general differential transfer function be 
available within a process of the following type: 

hk (^71 ^ • * *? ^ln s 1? * * •> £/n) (1) 

where k — 1, 2, ..., n and G k are continuous limited functions 
in ...,rj n and | l5 allowing first partial derivatives 

with respect to f l9 ..., £ m , and being defined in the open field 

N Oh? • • ”> Vm • • *j £m)* 

rjx, are the process variables, considered as variations 
from the optimum prefixed values, and £ l9 ..£ m are the control 
variables acting on the process variables rj l9 . .., rj n . For this 
reason they must more exactly be considered as unknown 
functions depending on ... 9 rj n : 

^ 1 ^ 1 0? 1J * * * 3 tfn) ? • • * £m m 0? 1 ? • * • 5 V n) 

The end of the assumed problem shall thus be to find out the 


form of these functions £ l9 ..., £ m within a prefixed C class, by 
using the criterion of optimizing the operation efficiency of the 
control system. 

In general, it might be assumed that all the process variables, 
on the whole, be controlled by an equal number of control var¬ 
iables, that is to say that m — n. 

For the above reasons, it is clear that the control laws which 
are to be found, 

£l = £l(»h 

shall be those for which the optimal integrated process con¬ 
trol is carried out on established values (prefixed setpoints) 
with a rigid feedback line. The resulting control action shall 
be that for which the system may reassume the stable equilibrium 
position as quickly as possible, whenever one or more simul¬ 
taneous troubles shift it from such a position. 


Dynamic Programming 


For readers’ facility, the dynamic programming method is 
anticipated in order to obtain Bellman’s functional equations. 
The control action of fr (r = 1, 2,..., rri) must be such as lo 
minimize the functional 




*00 

0 


F(ih,-*lnlZn-~Udt 


( 2 ) 


defined in function class C(£ v ..., £ m ) and in class C t of variables 
Vn •••> Vm where F is a function having a previously determined 
form. The minimization of (2) is equivalent to optimizing the 
operation efficiency of the control system. 

Variable values in the limits shall be 

lbrf o =0 r ls( t o)~ r lsO Zr(to) = U 

for t/= co !?,(*/) = 0 {,(*/) = () 

where r = 1 , 2, ..., m and s = 1 , 2, ..., n. 

Assume that xj/ (rj m r] 20 , ..., rj n0 ) be the functional 7(f a , .... £ m ) 
minimized, that is to say 

•AOJioj ••• ^no)— min F(rj u ..,ri n ^ u .,.^ m )dt 

•a 1 > » %m J 0 (3) 

Then it shall also be 

j Eqn '(7) I * 


where S is a small positive number. 


•A (*7 1 o> ■■■>’!no) = min 

«1 . in 


F(f] 


r co 

J S 


,)d t 


* Eqn (4): 


(4) 







SOME CONSIDERATIONS ON OPTIMIZED INTEGRATED CONTROL 


According to Bellman’s optimality principle, the formula (4) 
changes into 


Eqn (5) 


and, if S is sufficiently small and if it is admitted that ^ be a 
continuous function, partially derivable within the range 0 < S, 
one obtains the following formula: 


Eqn (6) f 


where 0 (S) is a residual, about which it may be supposed that 
the limit [0 (£)/£] = 0 for 5 -> 0, and d K satisfies the formula 
0 < 6 k < S. It results, from (6), that the following formula, 
obtained in the limit -> 0, must be equally null: 


dxl/ 


FfalO* ***’ £l0» ■••5 0^ 


0 = min 

<al 5 Sm 

The formula (7) must be valid, whichever the initial time instant, 
thus: 


Eqn (8) 


which is fractionated into the following Bellman’s system: 

_9F " = o 

0£i k =i 3% 3£i 


QF_ + £ 3>A . 80 ft _ Q 


(9) 


with the initial statements: 

for * 0 = 0 J 7 k ( 0 ) = 1ko 4(0) = ?fco 

for t f =oo r] k (co)=0 4( co ) = ° 

and the function F, appearing in (2), of the following square 
positive definite type: 

n m 

F=E«^ 2 +Z^r ni) 

s = 1 r = 1 

Then, Bellman’s eqns (9) are specified as follows: 


Z <W + Z c rC+ Z 


01^ 


r = 1 Jk= 1 tyk 


Z Z ^ 

„a= 1 ^ = 1 


(7) 2c r ^+Z^^ = 0 0=1,2, 


(9 a) 


t=i d>lk 

From the second equation of (9 a) it results that are bound by 
relations: 1 „ ~ , 

1 - (r=l, 2, ...,m) (9b) 

= 1 


C " r ~ 2c r Jr nhr orj k 


Kn ‘ /c=l 8^7 /c 

formed by (ra + 1) differential equations in (m -f 1) unknown 
variables f ls ..., 

For the existence and unicity theorems of the solution t ft 
(rji,...,rj n ) of the system, see Bellman 1 . The general solution 
proceeding is to eliminate the unknown functions in 

order to obtain a single differential equation which allows us to 
specify the solution i \f (%,..^ n ). Then, by means of the last m 
equations, the m functions f 1# ..f m are specified algebraically. 
By associating the resulting functions f 1# £ 2 ,..with (1), it is 
possible to find out the time functions: 

'h= f h(0> •■■> ) 7 n = '7«W ( J0 ) 

that is the laws of process variables under the optimized contro 
action. 

Study of Linear Integrated Control of n order in an Open Field 
According to Letov 

A transfer function within the process of the following 
linear type is assumed 

n m 

hk= Z Z (/c = l, 2 , (la) 

«=i f=i 


where the existence and unicity of solution for £ r clearly appeal 
in an open field, as soon as the existence and unicity of solution 
of functional i p are assured. 

By replacing (9 b) in the first of (9 a), the differential equa¬ 
tion, providing that the function \j/ is known, is obtained. The 
equation is the following one: 

" , £ 1 / A 0«A X2 

+ ( |2 > 

k = l % U=1 /? = 1 - C /J u= 1 ur lv 


and simplifying: 


" , ” 6iir " , l £ ( A 0iA 

Z + Z qt - Z 4 Z C *c( Z 0 „ m 

s = l k=l 0r lk<z=l * k= 1 \r=l ur /r 


The solution shall be of the following type: 


(12 a) 


(13) 


$= Z Z 

fc=la=l 

and its coefficients /t ; .. a may be obtained by well-known methods. 

Calculating all the partial derivates it is possible, 

by means of (9b), to obtain the functions £ r (r = 1, 2, m), 

which shall be of the following type: 

{rEw.. (r = 1,2,..., m) (14) 

These represent the law of optimized integrated proportional 
control, with fixed band. Control proportionality results, 
dependent on the particular choice of functional F. 


* Eqn (5): 


^(*7l0» **- ? 7no) == min 




f Eqn (6): 

•••> ? ?no)= min 

p(n 


.U 

- 

1 Eqn (8) : 

0 = min 

|r(»?i 


F(fJ 10, •••j^noSClO* <?mo)^ + , / y O?10 

_ " a^r 

••*? Cm)d“ Z-v ^ 

* = 1 0r lk 


.’-HtdA] 


S+0(S) 


(5) 

( 6 ) 
(B) 
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Associating(14) with (la), the law of i/ s = Vs (t), (s = 1 , 

may also be determined. The same result might be reached with 
the help of Lagrange calculus of variations. 

The control system obtained by (14) is also stable, once it 
is possible to assure the positivity of function i jj in 

£ n 2 s = o 

s — 1 


This is obtained according to Letov’s point of view 2 . Then this 
function \j/ coincides with a Liapunov’s function. 
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DISCUSSION 


A. Letov, Institute oj Automation and Telemechanics , Moscow , 
U.S.S.R. 

The problem dealt with in the paper is quite interesting. However, 
when it is considered in a closed space it gives rise to another purely 
mathematical problem, the solution of which is a necessary condition 
of existence of the optimal solution found for g r (r — 1, ..., m) 

The function generating the solution $ r should be at least contin¬ 
uous on the boundaries of the closed space. 

In the simplest case r = 1, when the closed space of the problem 
is defined by the inequality | | < 1, the problem of the continuity 

ot the generating function is reduced to the solution of the Cauchy 
problem for a specially constructed partial equation following from 
Bellman’s method. 

This problem has a positive solution in the given simple case It 
would be extremely interesting to know the solution of the given 
problem in the case r > 1. " 


S. Kahne, U.S.A.F., Cambridge Research Laboratories, Bedford, Mass. 

To solve a problem in optimal control theory Dr. Marcacci has used 
Bellman s notion of dynamic programming. For linear systems with 
a quadratic performance functional, his procedure leads, in a straight- 
forward manner, to an optimal control law which is a linear combina¬ 
tion of the state variables. I interpret the author’s ‘optimized integrated 
proportional control, with fixed band’ to be a suggestion for a feed¬ 
back solution to the problem. It should be noted that if one or more 
states are not observable, the solution would have to be modified. 
I might suggest, therefore, that the class of problems for which his 
e qn (14) is a valid feedback solution is the class of completely observ¬ 
able linear systems with performance functional F of his eqn (11) 

^ he ca f e conside red in the paper resulted in a non-linear partial 
differential equation for which a solution could be found by inspection 
In the more general case where the system or the performance func¬ 
tional was of a different form it would seem that this solution guessing 
would be difficult indeed. I would appreciate the author’s comments 
about the possibility of extending his method to these more general 
situations. 
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Summary 

Problems are considered of constituting the controller action in a 
system with time lags of pulses in the plant and of signals in the 
feedback channels. The problems are complicated by random load 
and disturbances. Statements of the problems are given, and criteria 
of optimality based on Liapunov’s method of functions, worked out 
for systems with after-action, and brought up to date in accordance 
with the principles of dynamic programming. An explicit form of 
optimal control is derived for the problem of the analytic design of a 
regulator to minimize a quadratic functional; in this example optimal 
control is put together from a linear functional of the prehistory of the 
process and a term determined by the actual value of the load. Approxi¬ 
mate methods of calculation of the optimal control signal are dis¬ 
cussed, and generalizations of the problem are indicated. A class of 
systems with incomplete information is indicated for which the problem 
of the analytic design of a regulator can be broken down, thus: (1) opti¬ 
mal control is found for a similarly determined problem; (2) The best 
prognosis of the state of the plant and load is established according 
to the given feedback channels; (3) the prognosis of (2), introduced 
into the law of control (1), gives optimal control. 

Sommaire 

II s’agit des problemes lies a la generation d’une action de commande 
dans un systeme dans lequel il existe des retards entre les impulsions 
donnees a Installation et les signaux cheminant le long des circuits 
de reaction. L’existence d’une charge et de perturbations aleatoires 
complique les problemes. La communication precise les enonces des 
problemes ainsi que les criteres d’optimalite; ces derniers sont etablis 
a l’aide de la methode des fonctions de Liapunov, adaptee aux sys- 
temes a action residuelle et modernisee conformement aux principes 
de la programmation dynamique. On en d£duit une forme explicite de 
commande optimale qui est adaptee au probleme de l’etude analytique 
d’un regulateur capable de minimaliser une fonctionnelle quadratique. 
La commande optimale resulte de la superposition d’une fonctionnelle 
lineaire de la prehistoire du processus et d’un terme dependant de la 
valeur effective de la charge. On donne des methodes permettant le 
calcul approche du signal de commande optimal et on indique la 
generalisation du probleme. On montre, que pour une classe de syste- 
mes a informations incompletes, le probleme de la synthese analytique 
d’un regulateur peut etre decompose en deux: (1) on conqoit la com¬ 
mande optimale par analogie avec un probleme similaire; (2) compte 
tenu des circuits donn6s de reaction, on fait la meilleure prddiction 
quant a l’etat de l’installation et quant a sa charge. Le fait d introduire 
les valeurs ainsi predites (2) dans la loi de commande (1) conduit a la 
commande optimale. 

Zusammenfassung 

Der Aufsatz behandelt den Entwurf von Reglern fur Systeme mit 
Zeitverzogerungen in der Strecke und mit verzogerter Ruckfiihrung. 
Die als regelloses Signal auftretende Last oder Storungen erhohen die 
Schwierigkeiten. Es werden die Probleme formuliert, optimale Bedin- 
gungen aufgrund der Methode der Liapunov-Funktionen fur Systeme 
mit „Nachwirkung“ ausgearbeitet und die Losungsform in Uberein- 
stimmung mit den Prinzipien der dynamischen Programmierung auf 
den neuesten Stand gebracht. Ftir das Problem des analytischen Ent- 
wurfes eines Reglers zur Minimisierung eines quadratischen Funk- 
tionals wird eine explizite Darstellung der optimalen Regelung abge- 
leitet. Hierbei ergibt sich die optimale Regelung aus einem linearen 
Funktional der Vorgeschichte des Prozesses und einem Ausdruck, der 


durch die wirklichen Werte der Last bestimmt ist. Naherungsmethoden 
zur Berechnung der optimalen Regelgrofie werden besprochen und 
Verallgemeinerungen des Problems aufgezeigt. Die Arbeit betrachtet 
eine Gruppe von Systemen mit unvollstandiger Information, fur die 
sich das Problem des analytischen Entwurfes des Reglers in drei Teile 
zerlegen laBt: 1. Zunachst sucht man die optimale Regelung eines 
ahnlichen Problemes. 2. Aufgrund der gegebenen Ruckfiihrpfade wird 
die beste Vorhersage des Zustandes der Strecke und der Lastgewonnen. 
3. Die Vorhersage von 2. in die Beziehungen der Regelung 1. einge- 
setzt, ergibt die optimale Regelung. 


Introduction 

The problem of forming the optimal process input for a regulator 
in a system with time lag of action and signals is considered in 
this paper. The questions considered belong to the class of 
problems of optimal control. These problems were first stated 
and developed in the U. S.S.R by Feldbaum 1 . The mathematical 
theory of optimal processes was worked out by Pontryagin 
et al . 2 , on the basis of their Maximum Principle. Their studies 
have given rise to a great number of works: for instance, 
that by Rozonoer 3 , and also the Theory of Dynamic Program¬ 
ming\ developed by Bellman and his colleagues on the basis of 
the optimality principle and the functional equations which 
follow from it, which embraces a very wide class of problem. 
Reference can be made to the authors whose works, among 
others, have a direct connection with this paper 5 " 18 . 

Reference can also be made to the works of those authors 
who, among others, have studied optimal control problems in 
after-action systems, and in more general systems with dis¬ 
tributed parameters 19 " 22 . 

The present work originates from the studies of Letov 23 * 24 , 
and the statement of the problem adopted here is a generaliza¬ 
tion, for systems with after-action, of the statement of the 
problem given by Letov 24 . The problems for systems with delay 
of the feedback signals considered below are related to problems 
of dual control 25 or of the theory of adaptive processes 26 . 

The solution proposed is based on the method of Liapunov 
functions and the theory of stability of motion 27 * 28 , developed 
for equations with time lags 29 , and modernized in accordance 
with the principles of Dynamic Programming 4 . Statements of 
the problems are given in this paper, and criteria of optimality 
and the principles of solution are formulated. For systems 
which can be described by a few actual equations, the explicit 
analytical form of the optimal regulator is given. Approximate 
methods for calculating optimal control are described, and 
problems complicated by random circumstances considered. 

Time-lag of Signals in the Plant 

Consider a controlled system (Figure 1) where z(t) is a 
controlled vector quantity at the output of the plant A, and £, a 
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scalar quantity, is the input of the regulator B, constituted 
on the basis of information on the actual error x = z - z° and 
possibly also on the actual values of the load rj (t) The special 
feature of the system is the time-lag of the signals in the plant A 
(Case I), or of signals in the feedback channels (1) and (2) 
(Case II), or of f in channel (3) (Case III). Each case will be 
examined separately. If Cases I—III are combined in one 
system, the statement of the problems and the solutions must 
be combined accordingly. 



0 ) 


( n t 0>] “ asympt0tjca lly stable 29 with respect to the disturbances 
*° (/ 0 + &) (to > 0) from a region 


and such that for all 
holds a minimum 


ll*°(0)||£G o (2) 

4 > 0 and a: 0 (t 0 + 0) out of (2) there 


• / [to>x°,‘f]=min ( 3 ) 

Here 

00 

£] = CO[t,X (; t, t 0 ,x°, £), {(t)J d t (4) 

where to is a given non-negative function, * (t, t 0 , x‘\ £) is the 
trajectory of (1) with initial conditions t 0 and x° (t 0 + 6) and 
a selected law of control £(/)=£ [t, x(t + 0)]. The control 
signa c can be constrained by a supplementary restriction 
f e w (far instance, | f j <; l). 


Figure 1 

Case I Assume that the disturbed motion of the system is 
described by the equation 

dx 

d7 x x • • • > * (t ~ h k ), rj ( t ), C] (1) 

where x is an «-dimensional error vector, h t is the time lag of 
signals in the plant (0 < h t < h, f = l,..„ *), / is a kjlown 

vector function of its own arguments, determined by the structure 
of the system, and v (t) is the load or disturbance. Besides this 
a functional determining the quality of the process is given and 

there may be a restriction on the magnitude of the control 
signal £. 

The disturbed motion x (f) of system (1) with after-action 
with t > t 0 > 0 is determined, as is well known, by the 
history x (t 0 + 6) (- h < 0 < 0) 0 f this motion. The initial 
function * Go + 6) (- h £ 0 < 0) will therefore be called the 
initial disturbances (with t = t 0 ). It is also convenient to con¬ 
sider as quantities describing the state of system (1) at instants 
f\f°’ and determining its future motion when r > t, sections 

L a tZV°Z S x(t + 0) (~ < 0). It is therefore 

suitable to form the control signal £ (r) at each instant / on the 

ba , S y IS [ °l mfor mation on the whole of the realized trajectory 

X J J + 6) Yf In other words, analytic con¬ 

struction of the regulator 24 means finding £ in the form of a some 

U . nC ‘°” a —Ht,x(t + 0)], determined on the curves 
x (t + 6) = {xj (? + 6), - 6 <: 0, i = 1,. n \ In future 

that the argument 6 varies the limits 

n 0 < 0. The continuous functions x (0) or x (t + 0) of 
die argument 0 are assumed to be elements of a certain space X 


II x (0) || = max (xj (0) + ... + x 2 „(d)f 
Also used is the notation 

11^(0)11 ==(ocf(0)+ ... + x 2 (0))* 

H*(01 = (*l(0 + ... +x 2 n (t))* 

Three problems are considered: 

motZ b t - n Find ? C T Wl Signal f = *° &*(*>! such that the 
motion x - 0 m a closed system (1), that is, with i (/) = [t, x 


rroDiem z. bind a control signal # = £<> [t, x (0)] assuring 
a minimum of B 

j t [4= x°, £°] = min J T [t 0 , x°, q] (0<t 0 <T) (5) 

where 

f r 

J T Ito. x °, £] = CO It, X ( t , f 0 , X°, £), { G)] df 

v fo 

+ iA[x(T, f 0 ,x°,£)] (6) 

and T< oo is a given instant of time, while ||x° (t 0 + 0)|| < G 0 . 

Problem 3. Find a control signal £ = [r,x(0)] assuring 

minimum of 

J c a [t 0 ,x 0 ,£,°']=vamJ K [f 0 ,x 0 ,<J] ( 7 ) 

where ||x° (r 0 + 0)|| ^ c 0 and 

co [ ? o> x °, <J] = lim when T-> oo (8) 

1 Iq 

In Problems 2 and 3, as in 1, it is assumed that the initial 
conditions x and trajectories x ( t, r 0 , x°, <f°) do not go beyond 
certain previously fixed regions. 

Tbesufficient conditions of optimality of the control signal <?° 
will be formulated for Problems 1 and 2. ‘ 

rWem/. JLet it be possible to indicate functionals 

V •*V/mi| ld * tA (0)], defined and satisfying in some 
region ||x (0)|| < G the following conditions: 

II mill Tbe bunCtl0nal v is P° siti ve definite with respect to 


* functl0naI v admits an upper limit with respect 

to||x(0)||. 

(3) The following inequality is satisfied: 
inf |>[f,x(0)] when ||x(0)|| = G, 

||x(0)|| = G]>sup[u[f,x(0)] when ||x(0)||^G o ] 

(4) Along trajectories of (l) 29 the derivative (du/dr), of the 
functional v satisfies the condition 

(Si + co ^ x ?°] = min fe) + co | J,x G), C]1 = 0 (9) 
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in the region ||x (t + 0)|| < G, and is negative definite with 
respect to \\x (OH in this region. 

Then £° [ t , x (t + 0)] is the optimal control signal for 
Problem 1, and the following equality is valid: 


The results of applying the proposed criteria to systems 
described by equations of actual form will be illustrated. 

Let the transient process be described by the linear differen¬ 
tial equations 


v [i 0 , X°(t 0 + 0)] = J [f 0 , x° Oo + 9), £°] (10) 

Note. Properties (1) and (2) generalize in a natural way the 
corresponding properties of Liapunov’s functions 27 that is (1) 
means that there exists a function w (r) > 0 with r ^ 0, such 
that v [t, x (0)] > w (\\x (0)||) with ||x(0)|| = ||jc (0)H, and (2) 
means that there exists a function W (r) satisfying the conditions 
W (0) = 0, v [f, x (0)] < W(\\x (0)||). If in Problem 1 the region 
G 0 encompasses any possible large initial disturbances x 0 
(the problem of optimal stabilization as a whole), the region G 
must coincide with the whole of the space X, and (1) is replaced 
by the condition 

lim v [t, x(0)] = oo when ||x(0)||-*oo, ||x(0)|| = ||x(0)|| (11) 

uniformly with respect to t. 

The demonstration of Theorem 1 is made by reasoning 
typical for the theory of stability of motion 29 , but taking into 
account the principles of dynamic programming 4 . 

The sufficient criterion of optimality for Problem 2 is for¬ 
mulated as follows: 

Theorem 2. Let there exist for every ||x° (t 0 -f 0)|| < G 0 and 
/ 0 6 [0, T) an admissible control signal £ (r), that is, a control 
signal for which the trajectory x (t, t Q , x°, £) may be prolonged 
in some finite region G until the instant t = T, and therefore the 
integral (6) is finite. If one can find in the region G functionals v 
[t, x (0)] and £° [/, x (0)] satisfying conditions (9), and 

u[T,x(0)] = ^[x( 0)] (12) 

then £° is the optimal control signal for Problem 2, and the 
following equality is valid: 

v [t 0 , x° ( t 0 + 0)] = J T [t 0 , x° (t 0 4- 0), c°] (13) 

The solution of Problem 3 can be obtained by passage to the 
limit from the solution of the problem when T~> oo. 

Note. If the load rj(t) is random or the system is subject to 
random disturbance, Problems 1 to 3 are modified as follows: 
integrals (4), (6) and (8) are replaced by their mathematical 
expectations (the conditional mathematical expectations for 
the appropriate initial conditions t 0 , x°, rj°) 9 and in Problem 1 
the requirement of stability is replaced by the requirement of 
stochastic stability 30 . In this case seek the control signal £° in 
the form of a functional £° [t, x (t + 0), ?] (t + r)], where 
— h < 0 < 0 and — h* < r < 0, while h* > 0 is the value 
of the maximal after-action for the probability process rj (t) 
(if rj (t) is a Markov process, then h* = 0). The criteria of 
optimality given above preserve their form, with the modifica¬ 
tion that v must here also be a functional v [/, x (0), rj (r)], 
and the derivative (dv/dt)% is replaced by its average value 30 
(dM{v}/dt)^. 

Conditions (9) reduce to partial derivative equations of 
a special kind. The solution of these equations in the general 
case is cumbersome; it is possible, however, to indicate a number 
of cases when an explicit form can be found for the optimal 
control signal, or when a numerical procedure for its deter¬ 
mination can be indicated. 


d X; 

d t 


~= X */(*)+ X CijWxjit-fy + bg + airiit) (14) 


j= i 


j=i 


where a ij9 c (j , a t and b it are known functions of time or constants. 
First assume that ij it) = 0, and then consider Problem 1 for 
system (14), assuming that 




m oo 

J to 


X x i (0 + ^ 2 (0 


Li= 1 


d t, 


2>0 —const 


(15) 


any initial disturbances x° ( t 0 -j- 0) are admissible. 

Here the functional v from Theorem 1 must be chosen in 
the form 


L'[f,x(0)]= ^ [du(t)Xi(0)xj(0) 
i,j= 1 


+ 2x f (0)J°^ /] i j(t,0)x j (8)d0 


+ f [ yij(t,0,r)x i {9)x J (T)dedT'] (16) 

J -hj — h 


which generalizes in a natural way the Liapunov function 
widely used in stability theory, as a quadratic form. If for every 
initial condition x°, / 0 there exists an admissible control signal 
£(r), that is, a control signal £(/) for which integral (15) converges 
uniformly with respect to t 0 , then there exists a functional v (16) 
satisfying the conditions of Theorem 1. From this it is directly 
concluded that in this case there exists an optimal control 
signal |° having the form 


c° [f,x(r+3)] 


= Z 


ll i (t)x i (t) + 


Vi(t , &)x t (t + 9) d«9 


(17) 


Conclusion 


The optimal regulator £° in system (14) with condition of 
minimum (15) is seen to be the regulator B , which applies to the 
input of the controlled plant A at every instant t a quantity 
£° (17), worked out on the basis of a measurement of the 
error x at the given instant of time t and at previous instants 
t — h < r < t, while the results of measurement of the previous 
errors x (r) = x (t -j- must be processed in the integrators 
J \ (t, •&) Xi ( t + #) d^. The control signal |° depends linearly 
on x (t + 0 1 ) (— h < & < 0). 

It is interesting to observe that for a system (14) with discrete 
delay h > 0 the optimal control signal must be worked out by 
an element with continuous distribution of the after-action 
Vi (r, #) over the whole of the time-lag interval — li < # < 0. 

Now let rj ( t ) =£ 0 be a known function of time. Consider 
for system (14) the problem (2), where 


frr n 

- E x ? 

J to Li = 1 


(t) + XQ 2 (t)\dt+ X Xj(T) (18) 

J i, j = 1 

Here any restricted control signal £ (t) is admissible, and 
the following assertion is valid: a functional v satisfying the 
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conditions of Theorem 2 exists, and differs in form from the 
functional (16) by the term 

" P° 

v* = £ (<5; (0 (0) + 9i (t, &) x, (9) dS )+e 0 t ) (19) 

i-1 J —h 

From this assertion follows the conclusion that in this case 
an optimal control signal always exists, and differs in form from 
the control signal (17) by a term £* = « (/) which is a function 
only of time t. 

Note. The conditions for Problem 1 solvability for sys¬ 
tems (14) and (15) reduce to the possibility of constructing an 
admissible control signal f (/). Here, as also in the case of 
systems without delay, the question is connected with the 
conditions of controllability of the system 11 * 31 . System (14) 
(with rj (t) = 0 ) will be called fully controlled in the interval 

hi (h > t 0 + h) provided that for every initial condition 
* + 0 ) there exists a continuous (piece-wise-continuous) 

control signal ((/) such that x (t, t 0 , x°, f )=0 when - h < 
<t<ti. The conditions of controllability, as in the case with¬ 
out delay 31 , can be investigated starting from the ‘L problem’. If 
system (14) is fully controllable in every sufficiently long section 
of the t axis, then it is optimally stabilizable in the sense of 
Problem 1. It is also observed that such stabilization is certainly 
possible if system (14) is asymptotically stable with f — 0 , or if 
the delay h > 0 is sufficiently small (or if the c id are small), and 
for the system dx t /dt = Za ij x j + the conditions of full 
controllability are fulfilled: the vectors {b,}, {\\a i3 \\ {£*}}, 

\Oi\s are linearly independent. The conditions 
of solvability of Problem 1 for (14) and (15) can also be ascer¬ 
tained in the process of solution, if the solution is sought by 
passage to the limit from the solution of Problem 2 for (14) 
and (18) (with yi iS = 0, rj = 0 and with T-> oo), which is 
sometimes a convenient method in practice. 

Now consider Problem 3 for system (14): accept that in ( 8 ) 

m = £ xf + l f 2 and assume rj (t) to be a random Markov 

i = 1 

function (for definiteness, of the pure discontinuous or diffusion 
type). Moreover, assume that system (14) is subject to some 
irregular disturbance of the white noise type, causing diffusion 
spread of x (t) in the time d/ with a matrix of second moments 
||M{d^ dx 3 -}|| = |k,.(t)dr||. 

The following result is obtained: if system (14) with rj(t) = 0 
is stabilized in the sense of Problem 1 , then an optimal control 
signal £° exists and has the form 

£° lt,x(t+6),n(0 ] 

» r po 

= .£ /kCOXjCO+J ^i(t,9)x l (t + 6)d6 +K(t,rj(t)') (20) 

It is interesting to observe that the first term here tallies 
with (17), and the random term k ( t , rj (t j) determined by the 
actual values of 17 (f) is the same as it would be if, with r > t, 
the function 17 (r) were determined and tallied with the prediction 
of its mathematical expectation M{rj (r)/rj (/)} made according 
to the actual value of rj (t). The magnitude of the dispersion of 
V (0 and the quantities a tj (t) do not affect g°, and manifest 
themselves only naturally in the quantity M{ / M [/ 0 , x °, tj 0 , | 0 ]}. 

As has been shown above, it is very laborious, in the general 
case, to construct the functional from Theorems 1 and 2 . The 


following methods may be indicated for its approximate deter¬ 
mination (and consequently that of £®): the small parameter 
method; approximate solution of the functional equation ( 9 ); 
approximating v in the mean; replacing the equations with 
delays or the functional equation (9) by finite difference equa¬ 
tions; replacing the equation with delays by a set of equations 
for the Fourier coefficients of a section of the trajectory 
x (t + 6), - h < 6 < 0 . These methods can be illustrated by 
numerical examples. 


Delay of Feedback Signals 

Consider now the system of Figure 1 when there is no after¬ 
action in the plant A, but signals in channels 1—3 can be 
delayed. 

Case II. Let the motion of the plant A be described by the 
vector differential equation 

dx 

-^=flt,x{t),rj{t),^\ + <p (21) 

where x, rj, g, /have the same meaning as in the first part of the 
paper, and (p is a disturbance of the white noise type, giving 
rise to diffusion spread of x (t) in the time dr with the matrix 

II M {dx f dx,.} || 1 = || (f) H df ( 22 ) 

The problem is to minimize the quantities 

J t =M<\ (o[t,x (f), Z(t)]dt+ij/ [x (T)]l df (23) 
and <0 ' 

=lim —with T-* 00 (24) 

i r 0 

The peculiarity of the case in question is that information 
concerning the actual values of the error x (f) and load ry (f) are 
supplied by way of channels 1 and 2 with delays of h x > 0 and 
h 2 >0 (or either A, > 0 or h 2 > 0 ) respectively {h x < //, h, < h). 
In other words, assume that in the regulator B at the instant t 
in the closed interval [ 0 , T] the values of the actual quantities 
x (e — A,) and »; (t — /z 2 ), where ry (t) is a random Markov func¬ 
tion, are known. Also assume that the regulator B is capable of 
remembering up to the instant t the signal g (t T 6) worked out 
by it with — h <, d < 0. Denote the set of magnitudes x (f — /q), 
rj(t- hi) and £ (t + 6) (— h < 6 < 0 ) by y (f), and x (- h x ), 

V (~ hi), g (0) (— h < 0 < 0) by respectively y. The quantity 
y (l) makes it possible to compose a probability description of 
the plant A at the instant t. The quantities J T (23) and (24) 
with the chosen law of control g may be regarded as functionals 
with respect to y (f 0 ), that is, 

MU (d [f, x (f), { (f)] df + ^(T)| = / r [f 0 ,y° (f 0 ), (25) 

w to J 

^Y~=J m [to>y 0 (to),Q (26) 

It is therefore reasonable in this case to seek the optimal 
control signal f° as a function of y (t), that is, in the form of a 
functional 

C(f) = C[f,y(f)] (27) 
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Call the admissible control signals the set of such functionals, 
sufficiently regular to give a meaning to the solution of (21) with 
£ (/)of (27), and, possibly, constrained by supplementary re¬ 
strictions arising from the statement of the problem (for 
instance, | £ | < 1). Designate the set of admissible control 
signals by the symbol E. Now the problem can be formulated. 

Problem 4. It is required to find a control signal £° belonging 
to 3 which minimizes (25) for all y° belonging to Y 0 , t 0 > 0. 

Problem 5. It is required to find a control signal £° belonging 
to 3 minimizing (26) for all y° belonging to Y 0 , t 0 > 0. Here Y 0 
is some region of the components y given in advance. 

Denote by x (r, y° ( t 0 ), £) the random motion of the system, 
generated by the initial conditions y° ( t °) with a certain choice 
of the control law; moreover, assume necessarily, with t 0 — h < 
<t< t 0 , that the control signal £ ( t ) tallies with that £ (t 0 T 0) 
(t 0 + 0 = t) which is a component of y° (t 0 ). 

Now formulate the criterion of optimality for Problem 4. 

Theorem 3. It is assumed that for all y°(/ 0 ) belonging to Y 0 and 
0 < t Q < T there exists an admissible control signal £ ( t ) (or 
£=£ [t 9 y (/)]) such that (25) has a meaning, is finite, and almost 
all the realizations {x (t , y° ( t 0 ), £), ?j (/, y° (r 0 )), £ (t -f 6) 
(— h < 0 < 0)} belong to Y, where Y is a certain region of 
values of y. Let it be possible to find functionals v [t, y] and 
f° [t, y] satisfying the conditions 

(1) v[T,y(T)] = Mtylx(T,y(T),&]} (28) 

for all y (T) belonging to Y 

( 2 ) ^Lp}j^ + M 

= min ^J+M{co[t,x(t,y(t),0,?]} =0(29) 

for all y (/) belonging to Y and all t in the closed interval [0, T]. 

Then £° [/, y (/)] is the optimal control signal for Problem 4 
and v [t Q , y° (t 0 )] -= min J T [/<>, y° (to), f]. 

The solution of Problem 5 is obtained by passage to the 
limit from the solution of Problem 4. 

The results of applying the given criterion to a system 
described by equations of an actual form are illustrated. 
Consider Problem 5 for the system 

dx n 

X a u (t)Xj(t) + b t i + a t n(t) + 4> (30) 

at j=l 

with the condition of minimum (26), where 

= p <Oy(O x i(0*y(0 + ^ 2 (oJ| d< 

+ t hjXi(T)xj(T) (31) 

Uj = i 

The delays along both channels 1 and 2 are assumed to be 
equal to h > 0, and it is admitted that any initial deviations 
x° (t„ — h) and r/ (/ 0 — K) belong to (f] lt r/ 2 ). 

With sufficiently wide assumptions concerning the character 
of the Markov probability process rj (t) and with the condition 
of full controllability of the system dxjdt = Sa tj Xj + b-i 


the functionals v [ t , y\ and £° [/, y] satisfying criterion (21) can 
be found, and passage to the limit with T —> co can be carried 
out. Problems 4 and 5 can also be solved. In addition the 
following result is valid. 

Results 

The optimal control signal for Problems 4 and 5 stated with 
conditions (30) and (31) has the form 

£° [hT (0] = E J“i (0 x i ( t-h)+v [f, n (t- h)] 

i= 1 
"0 

+ e[t,ff]t[t+e] de ' (32) 

J ~h 

The term v is determined at every instant t with respect to 
the realized rj (t — h), but to calculate it one must know the 
prediction M{rj (r)/rj (t — h)} with r > t — h. 

Here the functional v [t, y (r)] has the form of the sum of the 
quadratic and linear functionals of (t — h) and £ (f + 0) 
with coefficients dependent on rj (t — h). 

Analysing the resulting solution £° the following conclusion 
is arrived at: the optimal control signal £° chosen here at every 
instant t is the same as would be obtained in a deterministic 
system and without delay of the feedback signals; however here, 
instead of the known quantities (t) of the deterministic system, 
their best mean square predictions M{x* ( f)/x (t— h), r) (t— h ), 
£ (t + 0) (— h < 6 < 0)} must enter into the control law, and 
the deterministic load rj (r) (r > t — h) is likewise replaced by 
the mean prediction M{t] (r)/rj ( t — h)}. 

Case III. This case reduces naturally to the previous one, 
and it is not considered individually. 

When several of the cases analysed are combined in one 
system, the statements of the problems, criteria of optimality 
and results are combined correspondingly. 

In conclusion it is observed that Case II can be included in the 
more general case when incomplete information is transmitted 
along the feedback channels 1 and 2. For it can be assumed indeed 
that at the instant / there are applied to the regulator B signals 
y {t) and £ ( t ), statistically connected with x (t) and 77 ( t ) (in 
Case II, {y (t), £ (0} = {x(t- h ), rj (t — h), £ (t + 0)}) and 
an optimal control signal depending on these signals can 
be constructed. The foregoing reasoning and conclusions are 
generalized to this more general case. The quality of the process 
depends on how much the processes {y (t), £ (0} and {x (t ), 
rj ( t )} are connected informationally, or, in other words, how 
far the processes {x (,t ), rj ( 0 } are observable 12 with respect 
to {y (/), £ (/)}. 
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Optimal Control of Systems with Distributed Parameters 

A. G. BUTKOVSKI1 


Summary 

Plants with distributed parameters are widespread in many branches of 
engineering. The problem of designing control systems which are the 
best in a given sense for such plants requires the development of new 
techniques of optimal control. Control systems with distributed para¬ 
meters have a number of specific properties that make investigation in¬ 
to them very complex compared with lumped-constant systems. This 
paper gives a statement of optimal control problems on systems with 
distributed parameters for a fairly broad class of systems described by 
non-linear integral equations under arbitrary constraints. It formu¬ 
lates a maximum principle for this case, which gives the necessary 
conditions for determining optimal controls. 

The paper also considers approximate methods of solving optimal 
control problems on systems with distributed parameters, based on 
approximating the equations for the distribution functions by ordinary 
differential equations. 

Sommaire 

Dans de nombreux domaines de la technique on rencontre des installa¬ 
tions avec des parametres r^partis. Le probleme de l’etude des systemes 
de reglage de telles installations qui seraint les meilleurs dans un 
certain sens, necessite le developpement de nouvelles m6thodes 
d’optimalisation. En effet, les systemes de reglage avec des para¬ 
metres rdpartis ont certaines propriety qui rendent leur investi¬ 
gation plus complexe que celle de systemes de rdglage avec des para¬ 
metres concentres. Le rapport montre comment traiterl’optimalisation 
de tels systemes decrits au moyen d’equations integrates non-lineaires 
et certaines contraintes arbitrages. II formule un principe de maximali- 
sation qui donne les conditions de determination de systeme d’opti¬ 
malisation. 

En outre, ce rapport considere certaines methodes approchees 
d’optimalisation de systemes avec des parametres repartis, methodes 
basees sur une approximation des fonctions de distribution au moyen 
d’equations differentielles ordinaires. 

Zusammenfassung 

In vielen Zweigen der Technik finden sich Anlagen mit verteilten Para- 
metern. Die Schwierigkcit, irgendwie optimale Regelungen fur solche 
Anlagen zu entwerfen, macht die Entwicklung neuer Verfahren zur 
Optimalwertregelung notwendig. Regelsysteme mit verteilten Para- 
metern haben eine Anzahl besonderer Eigenschaften, die ihre Unter- 
suchung — verglichen mit der Untersuchung von Systemen mit zu- 
sammengefaBten Konstanten — erschweren. In diesem Beitrag werden 
die Probleme der Optimalregelung von Systemen mit verteilten Para- 
metern formuliert. Diese Formulierung gilt fur eine groBe Anzahl von 
Systemen, die durch nicht-lineare Integralgleichungen unter beliebi- 
gen Grenzbedingungen beschrieben sind. Fur diesen Fall wird ein 
Maximumprinzip angegeben, das auch die notwendigen Bedingungen 
fur die Bestimmung der Optimalregelung enthalt. 

In diesem Beitrag werden auch Naherungsmethoden fur die Losung 
von Optimalregelungsproblemen bei Systemen mit verteilten Para- 
metern betrachtet, die auf einer Naherung der Verteilungsfunktionen 
mittels gewohnlicher Differentialgleichungen beruhen. 


In many engineering applications the need arises for control of 
systems with parameters that are distributed in space. A wide 
class of industrial and non-industrial processes falls within this 


category: production flow processes, heating of metal in metho¬ 
dical or straight-through furnaces before rolling or during heat- 
treatment, establishment of given temperature distributions in 
‘thick’ ingots, growing of monocrystals, drying and calcining of 
powdered materials, sintering, distillation, etc., right through to 
the control of the weather. 

The processes in such systems are normally described by 
partial differential equations, integral equations, integro- 
differential equations, etc. 

The problem of obtaining the best operating conditions for 
the installation (the highest productivity, minimum expenditure 
of raw material and energy, etc.) under given additional con¬ 
straints has required the development of an appropriate mathe¬ 
matical apparatus capable of determining the optimal control 
actions for the plant. 

Pontryagin’s maximum principle and Bellman’s dynamic 
programming method have been the most interesting results in 
this direction for systems with lumped parameters. 

A wide class of systems with distributed parameters is 
described by a non-linear integral equation of the following 
form: 


Q(P)= 


D 


K[P,S,Q(S),U(S)-]dS 


(U 


Here the matrix 


Q(P) 


Q 1 (P) 

'or (Pi 


ne'(p)ii 


( 2 ) 


describes the condition of the controlled system with distributed 
parameters, while the matrix 


U(P) = 


u l (P) 
U r ( P ) 


= l|t/ i (i 5 )l! 


( 3 ) 


describes the control actions on the system. Here and in the 
following, the index i will refer to a row number and j to a 
column number in a matrix. The point P belongs to a certain 
fixed m dimensional region D in Euclidean space. 

The components of the single-column matrix 


K(P,S 9 q, U) = 


K 1 (P 9 S, Q, U ) 
K n (.P,S,Q, U) 


= 11 K l (P 9 S 9 Q, t/)|| 


( 4 ) 


belong to class L % and have continuous partial derivatives 
w.r.t. the components of the matrix Q. 

It will be assumed that the function U ( P ) is piecewise dis¬ 
continuous, its values being chosen from a certain fixed permis¬ 
sible set Q. Controls U ( P ) having this property will be called 
permissible. 
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Further, from the set of conditions Q (P) and controls U (P), 
related by integral eqn(l), let q functionals be determined,’ 
having a continuous gradient (weak Gato differential). 

I‘=I'[Q(P)], ( = 0,1,...,/ (5) 

r=/ i [e(n[/(P)] = ^(z), i = l+l,...,q ( 6 ) 

where 

z ° toeos), £/($)] ds 
9 /% 

z * F k LS,Q(S),U(S)-]dS 


F[S,Q(S),U(S)-] ds| 


The functions®^), /=/+1 ,.. q an d FHS, Q, U ), /=0,1..., k, 
are continuous and have continuous partial derivatives w.r.t 
the components of the matrices z and Q respectively. 

The optimal control problem is formulated in the following 
manner. 

It is required to find a permissible control U (P) such that 
by virtue of equation (1) 

f=0, i=0,l,.-,p-l,p+l,...,q (8) 

while the functional I p assumes its smallest value. Here p is 
a fixed index, 0 <p <,q. 

The following rectangular matrices are introduced 


s<z> 

i a®'! 

dz ~ 

dz J || 

dF 

laFll 

ae~ 

to 


i— 0,1,...,/; j — 0,l,...,k 
!=0,1,..., k; j=l,2,...,n 


grad/=||grad/||; i=l+l,...,q; j=l,2,...,n (11) 

where grad s I l denotes the yth component of the vector grad P 
w.r.t. the coordinate Q } . 

The following theorem 5 can be used as the basis of a solution 
of the problem formulated above on the optimum control of a 
plant with distributed parameters. 

Theorem. Let U = U (S) be a permissible control such that 
by virtue of eqn (1) the conditions (8) are satisfied and the 
matrix function M{P, K) = \\ Mij (P, R)\\, = 1,2,..., „, 

satisfies the integral equation [linear in M(P , R)] 

M ( p ,R)+~K[P,R,Q (j R ), U (1?)] 


of which at least one is not null, and also c p < 0, such that for 
almost all fixed values of the argument S e D the function 

n (S, U) = a [grad I {Q (P)}, K (P, S, Q (S), U} 

(P, R) K {R,S, Q(S), U} dP] 

0 r r 

+ b — <i> F {P,Q (P), U (P)} dP 

LJ D 

0 

• qqF{P,Q (P), u (P)}, k {P, s, 2 (S), u) 

- J D M (P, P) K {R, s, Q (S), 17} dP 

0 r r 

+ & 8i® ] D F i p >Q( p lU(P)}dP ■F{S,Q(S),U} 

(14) 

of the variable UeQ attains a maximum, i.e. for almost all 
S e D the following relation holds: 

n(S,U)=H(S) ( 15 ) 

where 

H (5)=sup k (S, V) (16) 

usSi 

As an example of the application of this theorem, consider 
the important practical problem of the heating of a massive 
body in a furnace. Let the temperature distribution along the 
* axis, 0 < x < L, at any instant /, 0 < / < p be described by 
the function Q = Q (x, t). Here the temperature U (?) of the 
heating medium, which in this case is the controlling agent, is 
a function constrained by the conditions 

A 1 <U(t)<A 2 , 0<t<T (17) 

i.e. in this case the set Q is the interval [A lt A 2 ]. 

It is known that the distribution function Q (x, t ), if initially 
zero, is related to the control V ( t ) by the following integral 
equation 


=J d m ( p> S ')qq K is, R, Q (P), u (P)] ds 

= L4^ [p ’ e (sx u(s) i m(s > r) ds ( i2 ) 

Then for this control, U (S), to be optimal there must exist 
one-row numerical matrices 

a = lko,q,...,c 1 || and 6=||c I+1 ,...,cJ (13) 


iC(x,tT)ir(T) d T (18) 

where K (x, t, r) is a known weighting function. 

In the heating of a body there is usually given a temperature 
distribution Q* = Q*(x) which is required to be attained in the 
minimum time. However, if the equation 

Q(x, t) ~ Q* (x) ( 19 ) 

for any permissible control is not satisfied for any fixed /, 
0 ^ t T, then the problem becomes that of determining a 
permissible control u ( t\ 0 < t < T, such that the functional 

/0 =J o [Q'*(x)-Q(x,T)Ydx (20) 

attains its minimum. Here y is a positive even integer. 

Since the integrand in eqn (18) is independent of the con¬ 
trolled function Q (x, t\ then according to eqn (12) the 
function M (/, c) = 0 for all t and r in the interval [0, T]. 
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It follows that the function n (r, U) takes the form 


71 (x, U)= C 0 


' L o_ 

o9 Q 


lQ*(x)-QY-K(x,T,x)Udx 


= -yc 0 U 


iQ*(x)-Q(x, T)Y~ l K{x,T,x)dx 


( 21 ) 


Since in this case by the conditions of the theorem c° < 0, 
so — yC 0 > 0, and hence the maximum of n (r, U) w.r.t. U, 
with A 1 < U < A^ is reached when 


U(r) = 


+ Aj 


( 22 ) 


A2 A 1 
+ 2 , sgn 


LQ*(x)~Q(x,T)J- 1 K(x,T,x) dx 


If we substitute expression (18) for Q (x, t ) in eqn (22), then 
we obtain an integral equation for determining the optimum 
control action U (r ). 

For example, if y = 2, A x = - 1, A 2 = 1, then the optimum 
control action satisfies the following integral equation: 


U (t) = sgn 



K (x, r, t) U (t) dt J K (x, T, t) dx 

(23) 


Opening the brackets and altering the order of integration, 
one finally gets 


U (t) = sgn 



*T 

N ( t , 6) U (6) dO 
o 


where N (r, 6) is the symmetrical nucleus 


(24) 


N(z,0) = 


'L 

K(x,T,T)K(x,T,6)dx 

o 


B( t) = 


L Q*(x)K(x,T, T)dx 
0 


(25) 

(26) 


Methods of approximating partial differential equations by 
finite difference equations can be applied successfully to the 
approximate solution of problems of the optimal control of 
systems with distributed parameters. This has the advantage 
that results obtained for lumped-parameter optimal systems 
can be used. 

As an example, consider the optimal control of a system 
described by the following equation 


e=e(M), o<x<s,o<t<T 

with these initial and boundary conditions 
Q(x,0) = Q 0 (x) 


m 

dx 


=a [17(0-2(0,0], 


dx 


= 0 


(27) 


(28) 

(29) 


Also let the function Q* = Q*(x) be given. The problem may 
be formulated in double form: 


(a) To find a permissible control U (/), 0 < t <T, Ue Q 
(Q is the set of permissible control values), such that the equation 

Q(x,T) = Q;\x\ 0 < x < S (30) 

is satisfied for a minimal time T. 

However, in many cases eqn (30) cannot be accurately 
satisfied for any T. It then makes sense to formulate the prob¬ 
lem as follows: 

(b) To find a permissible control U ( t ), U e Q, 0 < t <T, 
where T is a fixed time, such that the functional 

/ = j"[ e *(x)-2(x, T)]Mx (31) 

which characterizes the measure of deviation of the actual 
distribution from the given one (y a positive even integer), should 
reach a minimum. 

Using the straight-line method, problems (a) and ( b ) may be 
reduced to an ordinary problem of optimum control for systems 
with lumped parameters. 

In fact, splitting the interval [0, S] on the x axis into n equal 
parts by the points x 0 = 0, x 1 = s, ..., x n ~ S, where s — S/n, 
and replacing the second partial derivative of Q (x, t) w.r.t. x 
in eqn (27) by the second difference ratio, we obtain a finite 
system of order (n + 1) of ordinary linear differential equations 
for the functions qi ( t ), / = 0,1, 

q Q = —(^ (J J r p)q 0 -\.(jq l J r fjU 

Qi ~& (#i— i 2+ i = l,2,..., n 1 (32) 

4n=e(<l n -l-<ln) 
with the initial condition 

qi(0) = Q 0 (is) 7 i = 0,l.n (33) 

and the final condition 

qt(T) = Q*(is) 9 i — 0, 1,..., n (34) 

Here p and a are constant coefficients which can be expressed 
in terms of a and oc. 

In problem (a) the functional that has to be minimized is the 
time T. This problem can be solved by using the maximum 
principle. Gamkrelidze 3 has shown that its solution always 
exists and is unique. 

In problem (b) the functional to be minimized is 

I=i[di(T)-Q*(is)Y (35) 

i = 1 

In certain cases it is required to determine the optimum 
variation law for a control action which is itself distributed in 
space, constraints being placed on it in time and also in space 
coordinates. 

For example, it is sometimes material that too great space 
variations cannot be allowed in certain physical quantities such 
as temperature, pressure, electric field, etc. 

We shall consider as an illustration the heat-exchange 
equation 

b (y, 0 4 Q (y, t)+b (y, t ) v (t ) T Q (y > 0 ■+ Q (y> t)=u(y,t) 

6t 0y (36) 
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foi the exchange between a stationary heating medium with 
temperature U = U(y, t), 0 <y <L, 0 < t < T (y being a 
space coordinate and t the time), and a material moving at 
velocity v = v (t) ^ 0 in the positive sense along the y axis and 
becoming heated in the process of moving over the interval 
0 ^ y L. The state of heating of the material is described by 
the function Q (y, /). The initial and bondary conditions take 
the form 

Q(y>0) = Qo(yl Q(o,t)=o (37) 

In this case a permissible control is considered to be a 

function U = U (y, t), 0 < y < L, 0 < / < T, that satisfies 
the conditions 

A 1 <U(y,t)<A 2 (38) 

A 3^^U(y,t)<A 4 (39) 

where A„ A 2 , A s and A 4 are given constants. 

Physically these constraints correspond to the fact that in 
feed-through heating installations one cannot allow too great 
amplitudes of temperature fluctuation in the heating medium, 
or excessive temperature drop over the length of the furnace. 

In this case one has to determine the control U = U(y, t), 
subject to conditions (38) and (39), such that, in spite of all 
possible disturbances of the heating process caused by variations 
in the velocity v (t) and by variations in the thermal parameters 
^ C y> t) of the process, the deviation of the temperature of the 
material leaving the furnace from a certain given temperature Q * 
should be on the average a minimum, i.e. one has to minimize 
the functional 

^=f T [Q*-2(L,f)Pdt (40) 

Jo 

where y is a positive even integer. 

In order to reduce the partial differential equations to 
difference differential equations, split up the interval [0, L] on 
the y axis into n equal parts by the points y 0 = 0, y 3 = 1,..., 
y n — L, where / = L/n. Replacing the partial derivative w.r.t. y 
in eqn (36) by the difference ratio, we obtain a system of order n 
of ordinary linear differential equations in the functions q { ( t), 
/= 1 , 2 , 

b(il,t)q i +~b(il,t)v(t)[q i -q i _ 1 -] + q.= u t (t) (41) 

with q 0 (t) = 0, 0 < t < T and q t (0) = Q 0 (//). 

Equation (41) may be rewritten in the form 

— i = l,2,...,n (42) 

where £/„■ = U { (t) = U ( il , t), while the coefficients /S and 
can be expressed explicitly in terms of the functions v (t) and 
b (il, f). 

According to conditions (38) and (39) the function U t (t), 

0 < t < T, is subject to the constraints 


A i^U i (t)<,A 2 , i=l,2,...,n (43) 

A 3 l< U i+1 (t)~ Uj(t)<:A 4 l, i = l,2,..., n —1 (44) 

The functional (40) must now be replaced by: 

I= fjQ*-<ln(0] y dt (45) 

It can now already be seen that the maximum principle may 
be used for determining the optimum control actions Ui ( t ), 
1 ~ 1 5 2, .. n, 0 < t < T. In this case the permissible region Q 
from which the values of the control vector U(t) = U x {t\ 

U n (r) may be chosen is a closed convex polyhedron in n dimen¬ 
sional space, described by eqns (43) and (44). 

Observe that the function H (\j/ i9 U which has to be maxi¬ 
mized according to the maximum principle in this case for each 
fixed t 3 0<t<T ; takes the linear form in the U { 

J il/.U, (46) 

i=l 

Hence the problem of determining the optimum control 
actions Ui(t\ i = 1, 2, n, at any instant t, 0 < t < T, 
reduces to the linear programming problem of maximizing the 
function H while satisfying conditions (43) and (44). 

Therefore, besides the accurate and quite general methods 
of solving optimal control problems which have a wide 
application in engineering, great significance is also attached 

approximate methods of solving these problems, based on 
approximating partial differential equations by ordinary differ¬ 
ential equations. 
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DISCUSSION 


I. McCausland, Churchill College , University of Cambridge , England 

My question concerns the method of dealing with the end effects when 
a distributed-parameter system represented by eqns (27) and (29) is 
approximated by a lumped-parameter system represented by eqn (32). 


Consider a typical system represented by eqns (27) and (29), namely an 
electrical transmission line of length S with distributed series resistance 
and shunt capacitance, with the end a t x = S open-circuited. The 
voltage between the conductors is Q (x, t), and the end at x = 0 is 
connected to a generator of voltage U(t) through a resistance 1/a. 
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If the transmission line is subdivided into n sections in such a way 
that the voltages at a = 0, x = S/n, ...,x = S are explicitly represent¬ 
ed, it seems that each section should be represented by a lumped- 
parameter approximation as shown in Figure A , where R and C are 
the total resistance and capacitance of the line. If these n sections are 
connected together, the complete system is as shown in Figure B (in¬ 
cluding the input resistance 1/a). 
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Figure B 


It will be seen in Figure B that each end capacitor is half the 
capacitance of each intermediate capacitor. For this reason the first 
and last differential equations of this system are different from those 
of eqns (32) of the paper. For example, q n is double the value given 
by eqns (32), and q 0 is also different. In view of these differences, I 
would be grateful for the author’s comments on the suitability of 
different methods of subdivision. 


A. G. Butkovsku, in reply 

In deriving the finite difference approximations of eqns (27) and (29) 
in the paper I proceeded from a thermal interpretation of these equa¬ 
tions. The finite-difference system (32) was obtained by dividing a 
solid of thickness S into n equal layers, which were roughly considered 
to be thin bodies, in the thermal-engineering sense of the word. 

Compiling the elementary thermal balances of each elementary 
layer gives eqn (32). In the process, for the first (far left-hand) layer, 
account is taken of its interaction with the medium in accordance with 
the law of convective heat exchange. (The same is done for the last, 
far right-hand layer, which is simply insulated.) 

However, the difference scheme can, of course, be obtained by 
purely formal means from eqns (27) and (29), by breaking down the 
segment [0, S ] of the x axis, for example, into n -f 2 equal intervals 
by points x Q — 0, x t ~ s , x 2 = 2 s, ..., x w + 2 — $ where s = S/n + 2. 

Assuming that within each zth interval the temperature of the body 
is defined by the magnitude q%, i = 0,1,..., n + 1, and replacing the 
partial derivative in eqn (29) by the difference ratio and in eqn (27) the 
second partial derivative by the second difference ratio, we obtain 

— X — —— =a(u-q 0 ) 

s 


4 n + l -«n=0 

c h—-p.( c k-i ~ 2<?i + #i+iX i = l,2, n 

Expressing q 0 from the first equation and substituting (i — 1) in 
the third gives the following system in non-dimensional form: 


j qi = qi-i-2q, + q h i=2,3,...,n-\ 

1 Qn~ Qn- 1 Qn 

where 


• &Q at / « x 2r U 1+2/? 

<3 =~A t -j = (h + 2) 2 Fo, £ — 

uT 5 


l + p ,n 1 + [f'' X n + 2 

and Fo and Bi are the Fourier and Bio criteria for the given body. 




Bi 


D. P. Petersen, United Aircraft Corporate Systems Center , Farmington , 

Conn., U.S.A. 

This interesting paper broaches the important question of the control 
of dynamic systems with distributed parameters. The author lists in 
his opening paragraph some of the processes in which distributed 
measurement and control can be expected to be applied. I am engaged 
in the study and design of systems of weather information processing 
(not, as yet, of weather control), whence my interest arises. 

The use of the word ‘control’ in the title is perhaps misleading, 
since the paper deals only with the selection of an optimum input 
programme, distributed in space and time, to achieve or approximate 
the desired output (space-time) variation according to certain con¬ 
straints and criteria. The familiar and critical problems usually 
associated with control theory, (stability, response character and 
speed, etc.) are thus avoided. Although ‘disturbances’ to the system 
are mentioned, it is not clear how these are to be dealt with. 

Basic to any concept of control is the provision of means for 
measurement of one or more critical variables of the process. Without 
the recurrent flow of information from such measurements, logical 
operations are meaningless and disturbances and abnormalities cannot 
be compensated. In uni-dimensional systems (in which time is the 
only independent variable) and where a finite number of state variables 
is postulated, measurement feasibility is usually tacitly assumed in the 
theoretical analysis, and the design of suitable instruments is relegated 
to the component developers. In a multi-dimensional system, however, 
this question cannot be glibly side-stepped, since a spatial region 
cannot even conceptually be filled with an infinite density of measure¬ 
ment apparatus. Furthermore, economic considerations dictate even 
more sharply for such systems the necessity of seeking efficient dis¬ 
positions of measurement and control points in the space-time 
manifold. Thus, in my opinion, analyses based upon an assumption 
of essentially continuous measurement and control are not likely to 
lead to useful application. Such systems involve inherently and signifi¬ 
cantly discrete (multi-dimensional) sampling controllers. 

The question of sampling representations of multi-dimensional 
functions has been studied recently 1 * 2 . However, the approach thus 
far has been in an informational sense rather than from a control 
point of view. 

The following questions are asked: 

(1) Has the author investigated any problems of multi-dimensional 
feedback control ? 

(2) On what basis should the density of quantization [as discussed 
after eqn (40)] be chosen in order not to influence adversely the accuracy 
of solution for the optimum input programme? 

(3) Has any apparatus been constructed according to the theory 
discussed in the paper ? If so, does it involve periodic measurement of 
the output and re-computation of control inputs? What choice was 
made for the density and frequency of such measurements? Were 
various arrangements tried, either by simulation or by experiment, 
and what were the results of such variations ? 
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A. G. Butkovsku, in reply 

Dr. Petersen is quite right to spotlight the question of obtaining 
initial information about a distributed-parameter controlled plant. 
This is, in fact, one of the very difficult and important questions, both 
in the theory of optimal control of distributed-parameter systems, and 
in the practical realization of such systems. 

I must add that we also are greatly interested in these questions, 
and have had to face them in creating an optimal control system for a 
continuous billet-heating furnace 1 . In this case we began by building 
an adaptive analogue of a distributed-parameter plant 2 * 3 . 

All the main disturbance effects exerted on the plant were measured; 
they were fed to an analogue-digital model of the plant. The parameters 
of the plant were so adjusted as to minimize the deviation of the output 
parameter of the model from the value of the same parameter, as 


measured directly on the plant. After the period of self-adjustment, 
information on the state of the controlled plant was used in the con¬ 
troller, which realized the optimal control algorithm, found from 
theoretical examinations. 
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Solution of Optimum Control Problems by using 
Pontryagin’s Maximum Principle 

Y. SAKAWA and C. HAYASHI 


Summary 

This paper deals with the solution of optimum control problems, 
using Pontryagin’s maximum principle. Since an optimum control 
system which completes the transient response in minimum time be¬ 
haves like a relay servo, the important task of the controller is to 
reverse the sign of the manipulated variable at the proper instants 
according to the operating condition of the system. A set of points at 
which the sign of the manipulated variable is to be reversed forms a 
switching surface in the phase space. In this paper the optimum 
switching surface of a linear third-order system is investigated. Also 
dealt with is the optimum switching curve of a non-linear control 
system governed by van der Pol’s equation with an additional forc¬ 
ing term. 

Sommaire 

Ce rapport s’occupe de la solution des problemes de la commande 
optimale en utilisant le principe du maximum de Pontryagin. Puisqu’un 
systeme de commande optimale qui realise la reponse transitoire dans 
un minimum de temps fonctionne comme un asservissement a relais, la 
tache importante du regulateur est d’inverser le signe de la grandeur 
reglee aux instants appropries, selon les conditions de fonctionnement 
du systeme. Un ensemble de points, auxquels il faut inverser le signe 
de la grandeur regime forme une surface de commutation dans l’espace 
des phases. Dans ce rapport on recherche la surface optimale de 
commutation d’un systeme lindaire du troisieme ordre. On traite 
aussi la courbe optimale de commutation d’un systeme de commande 
non lineaire regi par l’equation de van der Pol avec un terme de 
contrainte supplemental re. 

Zusammenfassung 

Der JBeitrag beschaftigt sich mit der Losung von optimalen Regel- 
problemen unter Anwendung des Pontryaginschen Maximumprinzipes. 
Da ein optimales Regelsystem, bei dem der Ubergangsvorgang in 
minimaler Zeit abgeschlossen ist, sich im wesentlichen wie ein Relais- 
stellglied verhalt, ist cs die Hauptaufgabe des Reglers, das Vorzeichen 
der StellgroBe im richtigen Zeitpunkt, entsprechend den Arbeits- 
bedingungen des Systems, umzukehren. Eine Menge von Punkten, in 
denen das Vorzeichen der StellgroBe umgekehrt wird, bildet innerhalb 
des Phasenraumes cine Schaltflache. In dieser Arbeit wird die optimale 
Schallflache eines linearen Systems dritter Ordnung untersucht. 
AuBerdem enthalt sie eine Betrachtung tiber die optimale Schaltkurve 
eines nichtlinearen Regelsystems, das durch die van der Pol’sche 
Gleichung mit zusatzlicher AnregungsgroBe beschrieben wird. 


Introduction 

This paper treats the solution of optimum control problems by 
using Pontryagin’s maximum principle 1 . It is assumed that the 
magnitude of the manipulated variable is limited by saturation. 
In order to complete the transient response in minimum time, 
the magnitude of the manipulated variable is kept at its maximum 
value throughout the transient. This implies that the control 
system behaves like a relay servo. The important task of the 


controller is to reverse the sign of the manipulated variable at 
the proper instants according to the operating condition of the 
system. A set of points at which the sign of the manipulated 
variable is to be reversed forms a switching surface in the phase 
space 2 ’ 3 . 

In the following, the optimum switching surface of a linear 
third-order system is investigated and the optimum switching 
curve of a non-linear control system governed by van der Pol’s 
equation, with additional forcing term, is also dealt with. 


Optimum Control of a Linear Third-order Servo System 

The transfer function of this type of system is given by 

G(s)= s(s 2 +2Cs + 1) (1) 

where £ is a positive constant. It is assumed that the manipulated 
variable u of the system is subject to the constraint: 

M^l (2) 

The magnitude of the manipulated variable u must be kept at 
its maximum value during the control process 1 . Therefore, the 
controller must be of the relay type. The block diagram of the 
system is shown in Figure L The purpose here is to design the 
optimum controller which leads the system to a desired equili¬ 
brium state in minimum time. 
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Figure 1. Block diagram of a third-order servo system 


Assuming that the input signal 0< is a step function, the differ¬ 
ential equation of the system is obtained: 


d 3 e d 2 s ds 
"d? + “^ dt 2 + d t 


: 0 


(3) 


where e is the error signal. Eqn (3) is equivalent to the following 
set of linear equations: 

de x 


d t 


= *2 


de 2 

dF =£3 


dfij 
d t 


= s 2 2 £s 3 — u 


( 4 ) 
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where s 1 = e. Let the poles of the transfer function (1) be denoted 

;.i=-C+(C 2 -l) i ,A 2 =-C-(C 2 -l) i ,A 3 =0 (5) 

then Lurie’s transformation 4 for eqns (4) may be written as 
(assuming that 4= /.,) 


dxj[ 

— (5/tjXj + i 




2i(L 2 — Ai) X 2 (X 1 “"^ 2 ) 


X 1 , 

£2 -i^ + i^h 


^•i^i 2 ^2 

yl 2 — ^1 ^1 ^ L 2 


Upon applying the transform (6) to eqns (4), the following ca¬ 
nonical form of the equations is obtained: 

dxi . ) 

— =X 1 x 1 + u 


1 = a 2 x 2 + u 


The case where £ > 1 

In this case, X 2 and X 2 are real, both negative and distinct. In 
order to seek the optimum manipulated variable u 9 consider the 
Hamiltonian function 1 : 

H=(X 1 x 1 + u)\j/ 1 + (X 2 x 2 + u) i jj 2 + u\// 3 (8) 

where t^-’s (/ = 1, 2, 3) are defined by 

di J/g S H ' 

It nZ ~ 


dx 2 ~ 


Following Pontryagin’s maximum principle, the optimum manip¬ 
ulated variable u should maximize the Hamiltonian function 
of eqn (8) in which the other variables ^ and x t ( i = 1, 2) are 
considered to be constant. Hence, the optimum manipulated 
variable u subject to the constraint eqn (2) is given by 

u = sl S n OAi + ^2 +>A 3 ) = sign (i/h o e~ Mt + \f/ 20 e " Xlt + \j/ 30 ) 

( 10 ) 

where tj/ m , <j/ m and are initial values of the variables i fa, \j/ 2 
and \jj, at t = 0. It is clear from eqn (10) that the optimum 
manipulated variable u changes its sign twice at most. 

Take the desired terminal conditions x 1 — x% = x 3 = 0 as 
the initial conditions, and try to fill up the three-dimensional 
phase space with the optimum trajectories starting from the 
origin with decreasing time. Since u = ± 1, the following 
equations are obtained from eqns (7): 


where S = ± 1. Integration of eqns (11) gives 
SA 1 x 1 + l = C 1 Q dXlx > ] 


SA 2 x 2 + 1 = C 2 e^ 2 * 3 (12) 

A (5x z ) = At 

wheie C 2 and C 2 are constants of integration, and At represents 
an increment of the time. From eqns (12) the trajectory extending 
from the origin may be expressed by 

&*!*! +1 =e^ 3 | 

5A 2 x 2 + l=e dXlX3 \ 

There are two trajectories passing through the origin, one for 
u — d, the other for u ~ — d. The portions of the trajectories 
which tend to the origin with increasing time form a switching 
curve in the phase space. Consider the portion of the switching 
curve for u = d. Through every point on this portion of the 
switching curve passes a trajectory for u = — <5. A family of 
these trajectories forms a switching surface which terminates in 
the switching curve for u = d. The equation of the switching 
surface can be found, after a little calculation 3 , as follows: 


{1 + [1 — e " 1 * 3 (1 — 

= {1 + [1 — z sl2X2 (1 — <52 2 x 2 )]*} Al (14) 

A family of the trajectories for u = d reaching this switching 
surface fills up a half volume of the phase space; in other words, 
this surface is the termination of the family of trajectories for 
u = d whose initial conditions fill in the half phase space. 

When a step input is given to the system in equilibrium, the 
initial conditions are 

8i=^,e 2 =£3=0,orx 1 =x 2 = 0,x3= -A (15) 

where A is the magnitude of the step input. A point correspond¬ 
ing to the initial conditions [eqn (15)] is located on the axis 
in the phase space. A family of trajectories starting from points 
on the d x 3 axis forms a surface whose equation is given by 

(1+5X 1 x 1 ) X2 =(1+51 2 x 2 Y 1 (16) 

The phase portrait of the optimum third-order system with 
2 C — 2*5 is shown in Figure 2. In this figure, the trajectory 
O S 2 is the switching curve. The surface 5i O S 2 C is a family 
of the trajectories passing through the switching curve O S 2 and 
forms the switching surface whose equation is given by eqn (14). 
The surface A B O S l9 as given by eqn (16), is formed by a 
family of trajectories starting from points on the S x 3 axis. The 
straight line A S 1 in Figure 2 is a trajectory which starts from 
an infinitely distant point from the origin on the <5 x z axis. 
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Figure 2. Optimum switching surface and trajectories in the phase 
space (2 £ — 2-5) 


1 ... 1 



1 2 

3 


4 



s t = 2x 1 -x 2 -x 3 
s 2 = — x l + x 2 
£ 3 = “-*2 


( 17 ) 


Upon applying the transformation (17) to eqn (4), the following 
set of equations is obtained: 


dx 1 
d t 


— X\~\~ 11 


dx 2 
d t 


x 1 — x 2 + u 


dx 3 

dt 


— 11 


(18) 


Analogously to the preceding case, the optimum manipulated 
variable u takes its maximum absolute value under the constraint 
(2), and changes its sign twice at most. From eqns (18) there 
is obtained 

doc x _ 5x 1 — 1 

dx 2 Sx | + ox 2 — 1 


dx x 

dx 3 


— <5xi H-1 


dx 3 

dt 


= 5 


) 


(19) 


where d = ± 1. Each equation of (19) can be integrated 
separately. Thus 


l-& 1 = C 1 e" taj 

Sx 2 = (Sx 1 - 1) (C 2 + log 11 -(SxjQ 


( 20 ) 



Figure 3, Step response of an optimum third-order servo system 
(2C = 2-5; 


A(8x z ) = At j 

where C x and C 2 are constants of integration. From eqns (20) the 
trajectory passing through the origin, i.e., the switching curve, 
may be expressed by 

1—5x 1 =e -, * J 

5x 2 =(5x 1 -l)log|l-5x 1 [ 

Proceeding in the same manner as before, the equation of the 
switching surface is obtained, i.e., 



2[l-(l + gx 1 )e~ fa3 ]* 
l-Cl-a + foOe-**’]* 


log {1 + [1 - (1 + <5x0 e-^]^] 


The third part of eqns (12) enables one to find the total time 
expended in moving between two points along any combination 
of trajectories. Figure 3 shows an example of the step response 
of the system with 2 t == 2-5. 


+rSr— 5 x 3=0 (22) 

1 +uX x 

The step response of the variables may also be obtained as in the 
preceding case. 


The case where f = 1 

If f = 1, then = 4 = 1. Therefore Lurie’s transformation 
[eqn (6)] is inapplicable. In this case use is made of the following 
transformation: 


The case where 0 < £ < 1 

In this case, l x and are complex conjugate, i.e. 

Xy = — C '+ j (1'- C 2 f, *2= -C~j( 1 - C 2 )* (23) 
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Lurie’s transformation (6) is again applicable. The variables x 1 
and x 2 are also complex conjugate and can be written as 


x 1 =y 1 + jy 2 ,X2=y l -jy 2 

Lurie’s transformation (6) in real form is given by 

h=yi-y-~y2-ys 

i 

£ 2 =->’2 

CO 

£3=“Tl+^-3 ; 2 

where 

®=( 1 —C 2 )* 


(24) 


(25) 


Integration of eqns (28) gives 

\j/ 1 — (\j/ l o cos cot — \jj 2 o sin cot) 

\j/ 2 = e ?t (\j/io sin cot + \l/ 2 o cos cot) 

x l / 3 = l l / 30 


(29) 


where ift 0 , \jj 20 and */r 30 are initial values of the variables ift, 1 p 2 , 
and ^ 3 . The optimum manipulated variable w which maximizes 
the Hamiltonian function (27) under the constraint (2) is given by 


u — sign OAi+th) 

where 

/?= — tan -1 (t/' 2 o/>/'io) 


(30) 


Applying the transformation (25) to eqns (4) yields 

dy 1 


df 

dyj 

d t 

dy 3 

dt 


= -Cy 1 -coy 2 + u 
=my l -tiy 1 


= U 


(26) 


The Hamiltonian function for the system of eqns (26) is given by 

H=z( — £y x —coy 2 + w) xj/i + (coyi — £y 2 ) ^2 + ul p3 (22) 

where i/'/s (/ = 1, 2, 3) satisfy the following equations: 

d^i 0H 


TT- 

#3 = „^ = 0 

dt 0y 3 


(28) 


Take the desired terminal conditions = ^3 = 0 as 

the initial conditions, and investigate the optimum trajectories 
which extend from the origin for negative time. Let t = — x 
(r > 0), then from eqn (30) the optimum manipulated variable it 
is given by 

w=sign {e”^cos(coT + /?) —C} (31) 

where C and are parameters dependent on the initial values 
i/cio, \// 20 and 1 j/ zo . For each set of the parameters, an optimum 
manipulated variable 11 and an optimum trajectory extending 
from the origin are determined 1 . In Figure 4 , the damped 
sinusoidal curve x = q~ Ct cos co t with £ = 0T is shown. To 
obtain a sequence of switching instants for a set of the para¬ 
meters C and ft the origin of the time t is shifted to the point A 
whose coordinates are given by r — ftco and x = C. The 
intersections S x , S 2 , S z , ... of the damped sinusoidal curve and 
the straight line A B give the switching instants. Let the length 
of the time intervals between the switchings be r 1? r 2 , r 3 , ... as 
indicated in the diagram. When the parameter C is fixed, the 
lengths of the time intervals r 2 , r 3 ,... are constant even if the 
other parameter ft is varied. 

From eqns (26), 

dy 2 = <oyi-Cy2 (32) 

dj>i -Cy 1 -a>y 2 + u 
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Eqn (32) represents logarithmic spirals around the point y* = h 
y 2 = d (o (<5 = + 1) in the y l9 y 2 plane. Figure 5 shows a projec¬ 
tion of the optimum trajectories in the d y l9 d y 2 plane, which 
are drawn for a certain value of the parameter C (= 0-26). The 
optimum trajectories consist of the arcs of the logarithmic 
spirals with centre at the point M x or M 2 . For this particular 
value of the parameter C, the first section of the optimum traject¬ 
ory extends, for the time interval r ls from the origin O to a point 
on the arc O N. From this point the second section of the 
trajectory (with centre at M 2 ) extends for the time interval r 2 , 



Figure 5. Projection of optimum trajectories in the d y L , Sy 2 plane 

(t = 0-1J 



Figure 6. Optimum switching surfaces and a trajectory near the origin 

ft = cm; 


then the third section (with centre at Mf) extends for the time 
interval r 3 , and so forth. It is clear from the third equation of (26) 
that the increment of the variable y 3 is proportional to the angle 
of rotation of the representative point around the point M x or M 2 . 

By varying both the parameters C and /?, the phase space may 
be filled up with optimum trajectories. Figure 6 shows the 
optimum switching surfaces and an example of the trajectory 
(shown by thick line) near the origin. In the diagram, the 
trajectory O P which extends from the origin is the switching 
curve, where the switching corresponding to the first intersection 




Figure 7. Step response of an optimum third-order servo system (t, — 0*1) 

S x of Figure 4 occurs. The representative point moves along the 
switching curve O P during the time interval r v A family of 
trajectories passing through the points of the switching curve O P 
constitutes a switching surface Q , where the switching correspond¬ 
ing to the second intersection S 2 of Figure 4 occurs. The repre¬ 
sentative point moves on the surface Q during the time interval r 2 . 
The surface R is also a switching surface, where the switching 
corresponding to the third intersection £3 occurs. The switching 
surfaces where the switchings corresponding to the intersections 
S 4 > S',... occur are omitted in the diagram. 

Figure 7 shows an example of the step response of the 
system with f = OT. 

Optimum Control of a Non-linear Second-order System 

In this section the optimum control of a non-linear system 
is considered 5 . The system is governed by 

d 2 * , 2 a dx 

^2 + £(x 2 -1)~^-+x = w (33) 
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Figure 8, Block diagram of an analogue computer set-up for the solution 
of eqns (33), (38) and (39) for negative time (e ~ 0*3 ) 


where e is a positive constant. It is assumed that the manipulated 
variable it is subject to the constraint: 



\u\^ 0-5 (34) 

If it = 0, eqn (33) has a single periodic solution to which all 
solutions tend, regardless of their initial conditions. In order 
to counteract this tendency and keep the system at x — 0, 
dx/dt — 0, the manipulated variable u has been introduced. 
u is to be chosen, subject to the constraint (34), so as to reduce x 
and dx/dt to zero in minimum time. 

Eqn (33) is equivalent to 

dx 


—jj-= — x—£ (x 2 — 1) y-\-u 

The Hamiltonian function for the system of eqns (35) is given by 
H=y\J/ 1 + {-x-e(x 2 -T)y J ru}\lr 2 (36) 



where the variables \f/ x and are defined by 
d\j/ x 0j H ,, ^ . . 

■dr = ~&r =(1+ ' 2 " CJ ' ) * a 

■dr = ~a7 =- ^ i+s(x 


(37) 


The optimum manipulated variable u which maximizes the 
Hamiltonian function (36) under the constraint (34) is given by 

u =0*5 sign xj/ 2 (38) 

Eliminating the variable ^ in eqns (37) gives 


Our purpose is to determine the optimum switching 
the x, y phase plane, at which the manipulated variabl 
its sign so as to lead the representative point from £ 
conditions to the origin in minimum time. Since eqns 
and (39) are hardly solved because of their non-line 
was made of an analogue computer to obtain the 
switching curve. 

Figure 8 shows a block diagram of an analogue 
set-up for solving eqns (33), (38) and (39) for negal 



Figure 9. Optimum switching curve and trajectories for a 
system governed by eqn (33) (e = 0-1 ) 
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Figure 10. Optimum switching curve and trajectories for a non-linear 
system governed by eqn (33) (e = 05J 


Namely, the computer was set up to solve eqns (33), (38) and (39) 
with 5 t — — t (t > 0), where r is computer time and t is 
problem time. The desired terminal condition x = y = 0 are 
taken as the initial conditions for the computer solution. The 
initial conditions \j/ 2 (0) and (d ij/Jd t) 0 may be varied by the 
setting of the potentiometers a and b. For each set of the initial 
values if/ 2 (0) and (d i/^/d t) 0 , an optimum trajectory starting 
from the origin is obtained. Figures 9 and 10 show the optimum 
switching curves and trajectories for e — 0T and e = 0*5, 
respectively. 
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DISCUSSION 


I. Flugge-Lotz, Division of Engineering Mechanics, Stanford Univer¬ 
sity, Stanford, California, U.S.A. 

My particular interest in this paper is based on the fact that in 1959 
Mih Yin and I investigated time-optimal control of third-order 


systems 


_ 1 

(s + y)(5 r +2(5 + l) 


with 


ICI<|1| 


We studied solutions in backward time and developed an exact 
procedure for control in forward time for the case C = 0, y = 0 and 
an approximate procedure for the general case. The control should 
be realized by digital means because of an iteration for obtaining 
the initial values of the adjoints. Our report 1 contains many more 
examples than the printed version 2 published in 1961. In that year 
H. Titus and I improved (shortened) the iteration procedure 3 . 

Did the authors of this paper try to realize the optimum time 
control (in forward time) of the investigated systems ? 
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Y. Sakawa, in reply 

Our work partly overlaps that of Professor Flugge-Lotz. We obtained 
the optimum switching surface of a linear third-order system and the 


optimum switching curve of a non-linear second-order system by using 
backward-time technique; that is, we tried to fill up the phase space 
with the optimum trajectories extending from the origin with de¬ 
creasing time. But, now that the optimum switching surface and the 
optimum switching curve are obtained, we can realize such time- 
optimal controllers by using non-linear function generators of the 
analogue or digital type. 

For example, the optimum switching curves as shown in Figures 9 
and 10 of the paper can be realized by using either a servo-type func¬ 
tion generator or a photoformer-type function generator. Furthermore, 
an optimum switching surface as shown in Figure 2 of the paper can 
be realized by using a two-variable function generator of the type 
z = f (x, y). 

What type of function generator is most effective and proper for 
the actual controller is another problem; it belongs to the computer 
problem and is discussed in detail in the fine book of Professor A. A. 
Feldbaum of the U.S.S.R., ‘Computing Devices in Automatic Control 
Systems’. As for myself, I have had no experience in realizing such 
controllers yet. 


E. Pavlik, Siemens and Halske, Karlsruhe, Lassallestr. 9, Germany 

The optimum switching function F ( e, e , e) is introduced. This func¬ 
tion F is not very good for practical application (noise). Have you 
made trials of approximation of optimal control with simple func¬ 
tions, for example, F ( e , e) ? 


Y. Sakawa, in reply 

In many cases, we can obtain the time derivatives of the error signal 
from the control system itself. But I agree with Dr. Pavlik that in 
practice his theory is applicable. 


345 







Y. SAKAWA AND C. HAYASHI 


Sun Jian, Institute of Mechanics, Academy of Science, Peking, People's 
Republic of China 

Professor Sakawa has presented an interesting paper but I consider 
that by his method it is very difficult to solve the synthesis problems 
with equations of higher order. We have methods which are based 
upon the conception of isochronous region and we solve such problems, 
in general, as follows: 

Let us consider a linear system with variable coefficients 

x = A(t)x+B(t)u ; t>0; ueU (1) 

It is well known that if U is closed and convex, then the isochronous 
region is bounded, convex and closed set. In my paper, which follows 
this discussion, it is further proved that the isochronous region is a 
strictly convex n dimensional set and continuous in both t 0 and T. 

Furthermore, if for any point z on boundary S of Q there is an 
allowed control u(t)eU (interior of U) satisfying either inequality 

( - grad g (z), A(t)z + B(t)u (t)) > 0 (2) 

or the equality 

A (/) z + B (?) u (t) = 0 (3) 

then the isochronous region is strictly monotonously increasing with 
respect to T. 

By these properties we obtain: 

Theorem 1. If Gt 0 (Q, T) is monotonously increasing and there is a 
u (t)e U which transfers the xG Q into Q, then there exists a unique 
zeS and a smallest T such that the control 

u(t) = sgn (- grad g (z), cj) (t, t 0 + T)B (t)) (4) 

is a time optimal control transferring x into Q, where f(t, / 0 ) is the 
inverse fundamental matrix solution of original system. Finally, every 
z on S determines such a time optimal control. 

This theorem indicates that Pontryagin’s maximum principle 
is also a sufficient condition for time optimum, if Q and U satisfy the 
monotonus condition (2) or (3). However, the inverse conclusion is 
not true. 

Based on the above-mentioned theorem, one can easily find the 
switching hypersurfaces in phase space as a finite parametrical repre¬ 


sentation. Various examples show that this theorem is useful for 
synthesizing optimal control. 

If we let T (x, t) denote the optimal response time from x at 
instant t into JQ, then T (x, t) is a single-valued function defined in 
controllable region of phase space, and we have: 

Theorem 2. If Q and U satisfy monotonous condition (2) or (3), then 
optimal response time T{x,t) is continuous, piecewise differentiable 
and almost everywhere satisfies the following partial differential 
equation , \ 

(a -> A (t)x+B(t)uj=-l ( 5 ) 

with the boundary condition 

T(x,t) |,=0 ( 6 ) 

This Cauchy’s problem has a unique solution definable by grad g (z). 
The required optimal control function is 

U(x, t) = sgn(^-~, B(i)j (7) 

The method of getting a solution for (5)-(6) has been found. 
Thus, the synthesis problem for linear systems with variable coefficients 
is solved completely. When the Q consists of only a single point, for 
instance the origin, the two theorems are still true with only a little 
modification. 

Both the above theorems and proposed methods may be extended 
to a synthesis of optimal systems with another integral functional 
criterion. These methods have been well verified by many examples. 

Reference 

1 Sung Jien, Han King-Ching Theory of analysis and synthesis of 
linear time optimal control systems. Shuxue Jinzhen 5, Nr. 4 (1962) 

Y. Sakawa, in reply 

I think Dr. Sun Jian’s method for synthesizing time-optimal systems 
is very interesting. This method of approach is different to mine, i.c. 
he has obtained a partial differential equation which the variable 
T (x, t) has to satisfy. Once this partial differential equation is solved, 
the equation of the switching curve is obtained directly. 

However, I am afraid that the solution of the partial differential 
equation will not always be easy. 
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Summary 

In this paper some new results concerning analysis and synthesis of 
time-optimal control systems are given. We assume that the motion of 
the controlled plant is described by linear ordinary differential equa¬ 
tions with variable or constant coefficients, and the final states of the 
system form a convex region of phase space. Geometric properties of 
the isochronous region of these systems in phase space and the facts 
connected with these which have fundamental meaning for the solution 
of the synthesis problem are discussed. Moreover the facts demon¬ 
strated which are formulated in theorems have also their independent 
interests. At the end of this paper is given the method of synthesis of 
the optimal control function for constant as well as with variable 
coefficients, and illustrative examples. 


Sommaire 

Dans ce rapport nous avons soumis de nouveau resultats a 1 egard de 
1'analyse et dc la synthese des systemes de commande k temps optimal. 
Nous supposons que lc mouvement de l’installations commandee est 
dccril par des equations diflercntielles lineal res ordinaires avec des 
coefficients variables ou constants, les etats terminaux du systeme for¬ 
mant une region convcxe de i’espace des phases des proprietes geo- 
metriques de la region isochrone de ces systemes dans l’espace des 
phases, ainsi que les fails’ y rapportant qui ont une signification 
fondamentale pour la solution du probleme desynthese, se discutent 
ici En outre, les faits demontres, formulas dans des theoremes, ont 
aussi lour int6ret independant. A la fin du rapport, nous presentons 
la methode de synthese de la fonction de commande optimale pour 
des coefficients constants ainsi que variables, avec des exemples 
explicatifs. 


Zusammenfassung 

Die Arbeit gibt einige neue Resultate fur Analyse und Synthese von 
zcitoptimalen Rcgelsystemen. Wir nchmen an, daB die Bewegung des 
Regelobjekts durch gcwohnliche linearc Differentialgleichungen mil 
variablen oder konstanten Kocffizienten dargestellt werden kann und 
die Endstadien des Systems cinen konvexen Phasenraumbereich b.lden 
Die geomctrischcn Eigenschaften des isochronen Bereichs d.eser 
Systeme im Phasenraum und die damit verbundenen Tatsachen, d 
fur die Ldsung des Synthcsenproblems grundlegend sind werden em- 
gchend behandelt; in Theoremen formuliert sind diese Tatsachen auch 
als solche von Inlcressc. Das Endc der Arbeit gibt die Methode fur 
die Synthese der optimalen Regelfunktion fur konstante und variable 
Kocffizienten mit Anschauungsbcispielen. 


Recently, theoretical investigations on optimal control P r ° cess “ 
have made great progress. The proof and the developmen of h 
so-called ‘maximum principle’ opened a new dlre ^° n f ° r 
study of optimal processes. But up to the present the synthesis 
theories of optimal system are far from complete. In general^the 
synthesis theory cannot be directly derived from 
properties of optimal processes, because it has not been able to 

find the reciuired boundary conditions. 

The Sect of this paper is to report briefly some conclusions 
and a method of computation of the synthesis theory of the 


time-optimal control system which are obtained by a detailed 
study of the geometrical properties of the isochronous region, 
under the assumption that the motion of the controlled plant 
is described by a system of linear ordinary differential equations 
with variable or constant coefficients. The concept of the iso¬ 
chronous region was for the first time introduced 3 for the analysis 
of limiting the time-optimal control system. This was also suc¬ 
cessfully used 6 to synthesize a time-optimal second-order system. 
The relation between the isochronous region and the optimal 
control function has been indicated 7 . 

In this paper some theorems concerning the isochronous 
region and the time-optimal control for the linear system with 
variable coefficients are given in Sections I and II, which form 
the foundation of the synthesis theories stated in Section III. 
At the same time these theorems also present some independent 
interest. 

Owing to lack of space in this paper, we have only reported 
some conclusions, and omitted the detailed proofs. 

I. Consider a controlled object, the movement of which is 
described by a system of differential equations with variable 
coefficients of the following form: 


A(t)x+B(t)u, t>t 0 (1) 

at 

where u = (a 1 , w 2 ,..., « r ) is a point of a cube U: M < 1, 
f — 2 ,..., r, in the r-dimensional Euclidian space (r < n). 

Let <P (t 0 , t) be the solution of the matrix equation 


d t 


= A{t)$, $(t 0 ,t 0 ) = E 


Then, if an ^-measurable vector function u (t), t 0 < t < t 0 
+ T with restrictions | u l (01 < 1> i = U •••» A is given, t e 
solution of eqn (1) satisfying the initial condition x (t 0 ) - *o 1S 


je(f)=<i>(to,0 


x n + 


= <p(t 0 ,t)x Q + 




$(t,0B(t)k(x)<1t, t 0 <t<t 0 + T 


All ^-measurable vector functions u (0 defined on the interval 
\U t n + T) with the restrictions | u l (01 ^ 1 wil1 be cal * ed adnus " 
sible control functions, and the set of all admissible contro 

functions will be denoted by V(t 0 , t 0 + T). 

Let u (0 6 V (f 0 > h + T ) and x (0 be a solution of eqn (1) 
corresponding to u (t) and satisfying the conditions x (t 0 ) *o 
and x (? 0 + T) = ^ Then we shall call x (0 a trajectory cor¬ 
responding to u (0 and connecting x 0 and x v 

Definition 1. Let G+ (x„, T) (G~(x 0 , T)) denote the set of all 
possible terminal (starting) points of the trajectories w t 
u (0 6 LOo> to AT) and with the fixed starting (terminal) point 
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*o at f 0 (r 0 + 2 "). This set will be called T-isochronous region for 
the starting (terminal) point x 0 at t 0 (t 0 +• T). If the terminal 
points form a set Q then G (Q, T) will denote the T-isochronous 
region which takes a point of Q as the terminal point. Obviously 

G(Q,T) = u G~(x 0 ,T) 

and each isochronous region generally depends on the choice of t 0 . 

Let Q be a bounded and closed-convex set defined by the 
inequalities 


the first kind in a finite time interval. Therefore we may assume 
that ii ( t ) is left-continuous. Let V s (t c , t 0 + T) denote the set of 
these controls, then obviously 

K 0o> t 0 + T)c:V( 0 ,t 0 + T) 

Definition 4. If for T 2 > 7^ > 0 

G~(x 0 , TJczG - (x 0 , T 2 ) 

G(fi,r 1 )«=G(fl,T 2 ) 


g i (x 1 ,x 2 ,...,x n ) = g i (x)< 0, i=l,2,...,p 


where gi is continuous and differentiable, and grad g% (x) 4 = 0 
on the boundary S of Q. Let the symbol g (z) = 0 denote the 
aggregate of the relations which determine S, and let grad g (z), 
z e S denote the normal vector of the supporting hyperplane to 
S at z not lying on the same side with S. This vector will be 
called the externally normal vector of S at z and its negative 
vector the internally normal vector. 

If x 1 belongs to G + (x 0 , T) ( G~ (x 0 , T )), then there is u (t) e V 
(/ 0 , t 0 + D satisfying 


x 1 — <P (t 0 ,t 0 + T) 


*0 + 


t 0 + T 


<P 1 (t 09 z)B(z) u (z) dz 


( 2 ) 


Xi = 0 (t 0 + T,t 0 ) 


*o + 


t 0 +T 


(t 0 J rT 9 z)B (t) u (t) dr 


(2 a) 

If x x belongs to G (Q, T) then there are u ( t) € V(t 0 , t 0 + T) and 
z 6 Q satisfying 


x 1 — <P(t 0 +T 9 t 0 ) 


z-h 


$ 1 (t Q + T 9 z)B(z)u(z) dz (3) 

J t 0 +T 


Let S ± (x 0 , T), S (Q, T) and G ± (x 0 , T) G (Q, T) denote the 
b@undary and interior of the isochronous regions respectively. 

We now introduce the notion of continuity of isochronous 
regions as follows : 


Definition 2. Let T 0 > 0. If there exists a S > 0 for any e > 0, 
such that for any T l9 T 2 e [t 09 1 0 + T 0 ] with \T X —T 2 \ < d, we 
always have 

sup inf ||*i— x 2 l <s 

X 2 e 0 -k (*^o, T 2 ) x i e G (X 0 , Ti ) 


sup inf ji*i-* 2 |<e 

x t 6 G ” (^’Oj Ti) &2 e G ^ (^Oj T 2 ) 

the G ± (x 0 , T) is said to be continuous in T on [t 0 , t Q + T]. 
Similarly, we can define the continuity of G (Q, T) in T. 

Definition 3. Let T > 0 be an arbitrary number, yj 0 any non¬ 
zero vector and bt (t\ i = 1, 2, r, the zth column of B ( 1 t ). 
The control u ( t ) with the function 

if (t) = sgn (\l/ 0 , <P (t 9 1 0 + T) b t (0) 

t 0 <t<t Q + Z i = 1 3 2 ,...,r 

as its zth component is called a control determined by the vector 
yj 0 on [t 0 , t 0 + T] and denoted by 


k( 0 = sgnOAo, $(t, t 0 + T)B (0) , t 0 <t<t 0 + T 


then G~ (x 0 , T) and G (Q, T) are said to be monotone in T. 

II. In this section we shall give the fundamental geometrical 
properties of isochronous regions, and the existence and unique¬ 
ness theorem for optimal control, and a sufficient condition 
of the time-optimal control based on these properties. 

Let the system (1) be normal, then for T > 0, G ± (x 0 , T) is 
an zz-dimensional bounded closed strictly convex set which 
changes continuously with the time. Let x x e G ± (jic 0 , T) be a 
point to which the control u(t)e V (z t 0 , / 0 + T) corresponds, 
then for x 1 to be a point on S ± (jt 0 , T), it is necessary and suffi¬ 
cient that 11 (t) e V s (t Q , t 0 + T). Then u (t) e V s ( 4 , t 0 + T) is 
uniquely determined by x l9 the non-zero vector f 0 of u (/) is the 
externally normal vector of S+ (x 0 , T ) at x l9 and the vector — &* 
(t 0 , t 0 -f T)yj 0 (@* denotes the transposed of <P) is the externally 
normal vector of S~ (x 0 , T) at jc x . 

By using the continuity and the strict convexity of G ^ (a* 0 , T) 
for T, and the uniqueness of the control u (t) e V s (t Q , t 0 -f T) 
which transfers x 0 into x ± e S + (x, T), we can prove the following. 


Theorem 1. Let Q c= R n be a given bounded and closed- 
convex set, and x 0 e Q be a given point. If the systems (1) can 
transfer x 0 into Q by u (t) e V (f 0 , t 0 + T), then there exists one 
and only one control u* (t)e V s (/ 0 , to + 0 < J* 5 ' < T 

which transfers x 0 into Q by the shortest time T*. 

For the isochronous region G (Q, T), we can also prove that 
it is an w-dimensional bounded and closed-convex set which 
changes continuously with the time. Let x x e G (D, T) be a start¬ 
ing point of a certain trajectory x ( t ) satisfying the condition 
x (t 0 T) — z, ze Q, and corresponding to control u{t)E V 
Oo> *0 + T), then for x x e S (12, T) it is necessary and sufficient 
that z 6 S and u (f) e V s ( t 0 , + And then the vector which 
determines u (t) e V s (t 0 , t 0 + T) is some - grad g (z), where z 
is uniquely determined by the starting point x l9 hence u (t) is 
uniquely determined by x 1 and the vector 0* (/ 0 , t 0 + T) 
grad g (z) is the externally normal vector of S (Q, T) at x v Con¬ 
versely, let 'ipo be an externally normal vector of S (Q, T) at x l9 
then the vector 0- 1 * (t 09 t 0 f-T)yj 0 is an externally normal vector 
of S at z and —(/ 0 , t 0 + T ) yj 0 is a vector which deter¬ 
mines u (t) e V s (to, to + T ). 

Hence, when S is a smooth surface, S (Q, T ) is also a smooth 
surface. 

By the above properties, we can prove the following fact. 
Let u * (t) eJV s (t Q , t 0 + T) be a time-optimal control which 
transfers x 0 e Q into Q, x* (*), t 0 < t < t 0 + T, x* (t 0 + T) = z, 
be its corresponding trajectory, and z be the terminal point, 
then for any a > 0, 


z-f-a 


dx(t) | 


d t 


\t = t 0 + T-0 


eG + (x 0 ,7) 


Under the assumption of normality of the system (l) 2 , each 
component of u ( t ) has only a finite number of discontinuities of 


■ _d*(Q| 


d t 


E iQ 


|t= to + T-0 
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From this we can prove a necessary condition for time-optimal 
control which transfers x 0 e Q into Q. 

Theorem 2. For a control u (f) and its corresponding trajec¬ 
tory x ( t ) to be a time-optimal control and a time-optimal trajec¬ 
tory which transfer x 0 = x (/ 0 ) e Q into Q, it is necessary that 
T > 0, and a point z on S such that 


The meaning of synthesis-optimal control is to find a control 
function u ( x ) = (u 1 (x), u 2 (x),..., u r (x)), x e M, which, ex¬ 
pressed in terms of the position of points in M, is such that 
the trajectory x ( t ) of the system 

< ^-=Ax + Bu(x) 


u (t) = sgn (- grad g (z), 4> (t, t 0 + T)B (f)) 


starting from x 0 e M reaches Q in a certain time, and where 


( - grad g (z), A(t)x (t) + B(t)u (0)Ur o +r-o ^ 0 

gW=o 

We then have for any t, t 0 < t < t 0 + T, 


(4) 


(- <P* (/, / 0 + T) grad g (z), A (t) x(t) + B (i) u (f)) 


= C + 


to+T 


(-&* (T, to + T) grad g(z) 
/l ' (t) x (t) + B '(t) u (t)) dr 


where C is a non-negative constant. 

Nevertheless a control u ( t ) e % (f 0 , + T) which satisfies 

Theorem 2 is not always the time-optimal control from x 0 to A 
But we have a sufficient condition as follows. 

Theorem 3. If the isochronous region G ( Q , T) is monotone 
in T, then each control of V s (/ 0 , + 7 1 ) is a time-optimal con¬ 

trol which transfers any point of S ( Q , T) into A 

But when is the isochronous G (i2, T) monotone? We can 
prove a sufficient condition as follows: If for each zeS and 
— gradg(z) there is a control function u (t)e V (t Qi oo) satisfy¬ 
ing that (a) u(t)e U does not take values from the vertices 
of U, t > / 0 ; (b) one of the following two expressions holds: 


(— gradg(z), A(t)z+B(t)u(t))>0 9 t>t 0 
A(t)z + B(t)u(t) = Q, t>t 0 

then the isochronous region G (Q, T) is monotone in T. 

When Q is formed by an isolated point and is the origin O 
we obtain from Theorem 3 that when T> 0 the isochronous 
regions for any normal linear control system (1) take O as an 
inner point and are monotone in T, hence every control of V s 
0 () , t 0 + T) is a time-optimal control for the transfer of points 
from 5™ (0, T) to the origin O. 

III. In this section we shall give some results of the synthesis 
method of time-optimal control for systems with constant or 
variable coefficients. 

At first consider a normal system 


u (x (0) = (u 1 (x (0), U 2 (x (0), • ■ U r (x (0) 

is a time-optimal control for transferring x 0 to Q. 

Since, for the control system (6), each component u l of time-op¬ 
timal control u (; t ) takes only one of the values + 1 and — 1, the 
synthesis problem of time-optimal control is reduced to the fol¬ 
lowing for each component u l divide M into two parts and 
Ml such that u l {t) becomes + 1 within and - 1 within Mf. 
Let M- denote the common boundary of and M^. It can be 
proved that if S is smooth, then and Ml are connected open 
sets, and M ® = Ml n M", M — Ml u Ml. We call the 
switching surface of the system (6) for the zth component u l of 
u 0). Clearly, the points on the trajectory where u l (t) changes 
its sign are the points of AfJ. In order to solve the synthesis 
problem, we have only to find the switching surfaces. 

Let x e M 9 x g Q , and let T be the optimal time of response 
from x-to Q, then the time-optimal control u (t) from x to Q is 

u (t) ~ sgn (— grad g (z), e~ A(l ~ T) B ), 0 <t<T (7) 


where zeS. Clearly, when t = 0, the corresponding point on 
the optimal trajectory is x, and the value of time-optimal control 
is sgn (— grad g (z), e AT B). 

Hence 

u (x (0)) = sgn (■- grad g (z), e AT B) (8) 


Therefore, when t = 0, x (0) e S (Q, T) and the vector e A * T grad 
g (z) is the externally normal vector of (A T) at x (0), we have 
x e Ml when the inner product of bi and externally normal 
vector of S(Q f T) at x is positive; xe Ml when this product is 
negative; xeM\ when this product is zero. Hence we have 
parametric expressions of the switching surfaces as follows: 


x = e AT [ z-F 


e Az jB sgn (grad g (z), e B) dr 


(grad g(z), e AT bi )=0 
g(z) = 0, 0< T< co 


(9) 


^Ax + Bu ( 6 ) 

dr 

where A, B are constant matrices. The given bounded and convex 
set O satisfies the conditions of monotoneness (5). 

Let 

M= u G (A T) 

T^O 

then it follows from the boundedness, closure, convexity and the 
monotoneness of G (Q, T) that M is an open convex set contain- 
ing Q. A synthesis of time-optimal control, of course, can be 
carried out only within M. 


Solving the two components of z from the second and third 
expressions, and substituting them into the first, we get an ex¬ 
pression of Mg with n - 1 variables including the para¬ 
meter T. When n is small, this method is very convenient. 

If 72 = 2 the switching curve can be expressed by T; when S 
is a point this method is also useful. 

The following is another method for finding u (x). 

Let x G M, then according to Theorem 1 the optimal time 
of response T from x to Q is uniquely determined by x, hence 
it is a function of x, written T = T (x). Then we have T{z) = 0, 
z g Q. We can prove the following: 
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Theorem 4. Let G ( Q , T) be monotone in T, then T — T (x) 
is continuous in M. If S is smooth and satisfies (5), and if the 
set of those z which do not satisfy the first expression of (5) is 
only a surface on S whose dimension does not exceed n — 2, 
then T (x) is everywhere continuous and differentiable on M 
except at the points of i = 1, 2, r and those of S. Let m 
be a connected sub-region of M not containing any point of Af ?, 
/ = 1, 2, ..., r, then T (x) satisfies in m a linear partial differential 
equation of the first order 

(grad T (x), Ax + Buq)— — 1 (10) 

where u 0 is any vertex of U. 

If T — T (x), xeM was obtained, then the surface determin¬ 
ed by T (x) = T 0f T 0 > 0, coincides with S (Q, T 0 ) and the vector 
grad T (x), xe S (Q, T 0 ), is an externally normal vector of 
S (12, T 0 ) at x, namely grad T (x) is parallel to the vector e A * T 
grad g(z). Therefore we obtain from (8) 

u(x)=— sgn (grad T (x), B) (11) 

Thus the synthesis problem was completely solved. 

In order to solve T (x) from (10), we ought to determine first 
of all the value of u 0 . Let a set of points of S determined by 
(grad g (z), bi ) = 0 ,i= 1,2,.. .,r, be (n — 2)-dimensional hyper¬ 
surfaces, then S is divided by them into 2 r parts: *S l5 S 2 , ..., S 2 r, 
and on each Si the control function u (x) is completely determin¬ 
ed by (8). These values are denoted by u l9 w 2 , u 2r hereafter. 

Then by the Theorem 4, the monotoneness condition and 
the assumption made on S above, the Cauchy problem 

(grad T (x), A x + Bu t ) = -1 

0 

has a unique solution. We extend this solution T — T(x) in the 
direction of increasing t along the integral curve of the system 
of equations 

dx 

~U= —Ax — Bu i , x 0 eS i 

until (grad T (x), bj) = 0 for some j for the first time. Further, 
we take the surface (grad T(x), bj) = 0 as a new initial condi¬ 
tion for solving the equation 


plant be and suppose terminal states from a circle 
origin as its centre, and p its radius, defined by 

g(x, y)=x 2 +y 2 —p 2 <0 

where 0 < p < 1 , we can easily verify that the mom 
condition (5) is here fulfilled. 

Moreover, since any point in the phase space can b 
to the origin by an admissible control, the domain A 
whole phase plane are superposed, hence the synthes 
carried out within the whole plane. 

From (4) we have in the upper half-circumference 

(grad g(x,y),b) = 2y>0 

hence u = — 1 . Substitute in ( 10 ) and solve the part 
ential equation of first order 

dT ST < A ^ »i 

0 ^. y 0 y 1 0, T (x, y)\ x 2 +y 2 = p z 0 

A solution of the above-described Cauchy probk 

T b‘.y)->-j2j p > +1 - 2 (x+ 

The switching curve is determined by an algebraic eqi 



(grad T (x), Ax + Buj )=-1 

where Uj is different from in only by a sign of the yth com¬ 
ponent. Since, when S is smooth, it is impossible that there 
should exist a segment of any time-optimal trajectory on the 
surface determined by (grad 7Tx), bj) = 0 , Cauchy’s problem 
of this kind has a unique solution. 

Continuing in this manner we can obtain a solution T = T(x) 
within all M. Hence u(x) is completely determined by (11). 
Clearly the switching surfaces for are 

(grad T (x), b t ) = 0 , z*=l, 2 ,r 


Substituting in (10) the other value of u and solvii 
can obtain a switching curve on the lower half plane. ' 
phase plane is divided into two parts M + and by 
liquely symmetric switching curves M° and Q. The time 
control function can be written as 


u (x, y) = 


+ 1 > 
- 1 , 


{x,y)eM + 

(x,y)eM~ 


The results obtained above can be perfectly genen 
the system with variable coefficients. For the system 
parametric expressions (9) of the switching surfaces be< 


Thus the synthesis problem which is proposed here is 
solved. 

Example 1 . Let the equation of motion of the controlled 


dx 


d t 





\u\<l 


rt+r 


x = $(t + T 9 t)z + 


<P (t, 0 B (t) sgn (grad g(z') 


<P(T,f+r)5(T))dT 
(grad g (z), <P(t,t+T) b t (t)) = 0 
g(z) = 0 , 0 <r< 4 -oo 
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For the system (1), those switching surfaces depending on t 
are denoted by M 9 (0, and the two parts of M divided by M-(Y) 
are denoted by M^ 1 ( t ) and M7 (0* The necessary and sufficient 
condition for x (/) to be a time-optimal trajectory is that 
“I" 1, if x(/)GM>(r); «<C0 = — 1, if x (t) e MJ (t). 

Clearly the synthesis function u (x, t) is 


u l (x, t) - 


+ 1, xeM?(t) 
-1, xeMi (t) 


A partial differential equation of the first order which corre¬ 
sponds to equation (10) is 


07 07 
0x l ’9.x ?: 


+|=-1 


where 7 — T(x, /) denotes the time of optimal response start¬ 
ing from x at t to U. It is clear that 7(x, t) > 0 and T(x 9 f)\s= 0. 
In order to solve 7(x, /) from (10a), we divide first a cylindrical 
surface g (z) ~ 0 into 2 r parts in the (n + 1 )-dimensional space 
by the relations 

(grad g (z), bi (f))=0, /=1,2,..., r 

Then the values of the control functions on these parts are 
determined for t t. 

Thus we can solve 7(x, t) in a way similar to solving (10). 
It is obvious that if 7(x, t) was obtained, then 


«(x, 0 : 


07 07 


, ,B(t) 


Example 2. We shall find by virtue of (9 a) switching surfaces 
M ( l (/), i 1, 2, of the following system with variable coefficients 
which takes the origin as the terminal point: 

^ = * 2 + (l +e~ f ) li 2 


+ U + e“V\ |w 1 |<1 5 I^ 2 I<1 

at 

It is clear that the system is normal. Since the origin is 
the terminal point, the expressions (9a) of M^(t) become 


the expression of M\ (/) is 

rVcosT, -sin TV sgncosA _ 
Jo \sm L cosT/\-sgnsm x) 

After the integration is was carried out, we have for 
n n + l 


( (2« + l) + e ‘Y e ‘ 2 + sina-cosa —e 0 + 2' I+a )cosa 
1 = 0 

-e~ r Y cosa—sina—e~( t+2 ’ t+ “)sina 

i = 1 

where oc = T— njln (Figure 2). 

m8(oo) 

x2*\/M§(0) 


M 2«»/\L°(=o) 


(p (x, 0 B (t) sgn (i //, <P(x,t+T)B (i)) dr 


ty,<P(t,t+T)b t ( t)) = 0 
|i//| = l, 0<T< H-oo 

We solve from the second expression above % = ± (cos T, 

— sin T), f., ± (sin T, cos T). Owing to the symmetry, we 

have only to" find that part of M\ ( t) determined by % = (cos T, 

— sin 7). Since 

( cos (t t\ sin (r — t)\ 

$ (t, t) -1 sin ^ ^ cos ( T _ t y 


Figure 2 
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Most Recent Development of Dynamic Programming Techniques 
and Their Application to Optimal Systems Design 

R.L. STRATONOVICH 

Summary 


Dynamic programming theory will solve, in principle, many of the 
problems connected with optimal systems synthesis, but the number 
of complex problems actually solved with this method is not large, 
because the calculations become more and more difficult the more 
complicated the problem becomes. Introduction of ‘sufficient coordi¬ 
nates’, on which the risk function depends, eases the situation, the 
‘sufficient coordinates’ forming the space in which the Bellman 
equation is considered. In the paper a non-trivial example illustrates 
the effectiveness of introducing sufficient coordinates. In the example 
the sufficient coordinates are a combination of a posteriori prob¬ 
abilities and the dynamic variable of the controlled plant. 

A practical result of the introduction of sufficient coordinates is 
hat the optimal control breaks down in two units, one producing 
he sufficient coordinates, the other providing optimal control. 

Sommaire 

L’emploi de la theorie de la programmation dynamique pourra, en 
principe, resoudre beaucoup de problemes de synthese des systemes 
optimaux, mais jusqu’a present on n’a resolu que peu de problemes 
complexes parce que les calculs deviennent de plus en plus difficiles, 
selon la complexite du probleme. L’introduction de ‘coordonnees 
suffisantes’, desquelles depend la fonction de risque, ameliore la situa¬ 
tion, car celles-ci forment l’espace dans lequel on peut considerer 
1 Equation de Bellman. Dans ce rapport un exemple non courant 
montre 1 utilite d’introduire des coordonnees suffisantes. Celles-ci se 
composent dans l’exemple d’une combinaison de probabilites a 
posteriori avec la variable dynamique de Finstallation reglee. 

Un resultat pratique de l’introduction des coordonnees suffisantes, 
consiste a diviser la commande optimale en deux unites, dont l’une 
produit les coordonnees suffisantes, et l’autre assure la commande 
optimale. 

Zusammenfassung 

Die Theorie der dynamischen Programmierung kann grundsatzlich 
viele Probleme der Synthese optimaler Systeme losen. Sie wurde jedoch 
bis jetzt noch nicht haufig erfolgreich angewendet, da die Berechnun- 
gen nut zunehmendem Komplexitatsgrad immer schwieriger werden. 
Die Einfuhrung „ausreichender Koordinaten“ von denen die Risiko- 
funktion abhangt, erleichtert die Situation, wobei diese Koordinaten 
den Bereich angeben, in dem die Bellmansche Gleichung betrachtet 
wird. Anhand eines schwierigen Beispiels wird die Nutzlichkeit der 
Einfuhrung ,,ausreichender Koordinaten“ gezeigt. Hierbei sind diese 
erne Kombination von a posteriori-Wahrscheinlichkeiten und der 
Zeitveranderlichen der Regelstrecke. 

Ein praktisches Ergebnis dieses Verfahrens ist die Aufspaltung der 
Optimal-Regelung in zwei Teileinheiten: eine erzeugt die „ausreichen- 
den Koordmaten“, die andere fuhrt die Optimalregelung durch. 


Introduction: Block-diagram of an Optimal Controller 

As this is known 1 " 4 , dynamic programming theory solves, in 
principle, a large number of the problems connected with opti¬ 
ma systems synthesis. The applicability of dynamic programming 


methods is not impaired by taking into account white Gaussian 
noise and other random factors in various components—the 
statistical nature of the signal to be reproduced, imprecise 
knowledge of it, random influences on the controlled plant, or 
interference in the feedback circuit ( Figure /). Of course, as the 
problems grow more complicated, the actual performance of 
the calculations becomes more and more difficult. 

Although the basic principles of dynamic programming 
were expounded long ago, the number of non-trivial problems of 
optimal control theory actually solved by this method is not 
large. This is explained by purely computational difficulties 
which have to be overcome before a solution is found. 



Figure 1. Optimal servosystem. DK: sufficient-coordinates unit; OP: 
parameter-determination unit; OC: optimal control unit; PC): con¬ 
trolled plant 


What has been said confirms the importance of the develop¬ 
ment of new methods and techniques to increase the effective¬ 
ness of the theory and make it easier for concrete results to be 
obtained. 

In complex statistical problems the effective use of the 
theory becomes possible as a result of the introduction of 
sufficient coordinates’ on which the risk function depends. 
The importance of this concept was noted by Bellman and 
Kalaba , and the author has clarified and developed it further'-«. 

The sufficient coordinates form the space in which the Bellman 
equation is considered. A non-trivial statistical example is used 
in this paper to illustrate the effectiveness of the introduction 
of sufficient coordinates. In the example, the sufficient coordi¬ 
nates are a combination of a posteriori probabilities and the 
dynamic variables of the controlled plant. 

In complex statistical problems the introduction of sufficient 
coordinates has the result that the optimal controller breaks 
down into at least two consecutive units, each of which is 
constructed according to its own principles. The first unit DK 

(Figure 1) produces the sufficient coordinates X. In some 
dynamic programming problems it is trivial, but in complex 
statistical problems it may perhaps prove most important. In 


352 








MOST RECENT DEVELOPMENT OF DYNAMIC PROGRAMMING TECHNIQUES AND THEIR APPLICATION TO OPTIMAL SYSTEMS DESIGN 


the latter, it is synthetized with the aid of methods similar to 
those of non-linear optimal filtering 7 . In the example considered 
below, it simply coincides with a unit effecting optimal non¬ 
linear filtering. 

The signals from the DK unit output are sent to a further 
unit OC , which produces the optimal control action. The form 
of this unit, which converts the sufficient coordinates into a 
control signal, is found by consideration of the Bellman 
equation. This unit can be synthetized without great difficulty 
if the risk function is first found as a solution of the Bellman 
equation. The most difficult problem is obtain this solution. 
Therefore, techniques and methods, which make it easier to 
obtain the solution of this equation, are of interest. 

The equation is made far simpler by considering the station¬ 
ary mode of operation, when the time-dependence and time- 
derivative are eliminated from the Bellman equation. The corre¬ 
sponding stationary equation was considered by Stratonovich 
and Shmalgauzen 8 , and the method quoted is also described 
in this paper. Furthermore, to solve the resulting equation, use 
is made of the asymptotic step-by-step approximation method, 
first expounded by the author 9 . This method is convenient for 
the case of small diffusion terms, and makes it possible to 
obtain consecutive approximations whose accuracy is deter¬ 
mined by the magnitude of the coefficients for the second 
derivatives in the Bellman equation. 

It must be noted that the number of methods for approximate 
solution of the Bellman equation, which can be thought of for 
the solution of concrete problems, is practically unlimited; each 
method is best suited for the solution of problems of a particular 
type. To these should be added the method for obtaining of a 
solution on analogue or digital computers. Out of the whole 
range of methods, a special approximate method will be des¬ 
cribed and applied to the example under consideration, in the 
concluding part of the paper. The essence of this method is that 
the risk function is represented as a function whose appearance 
is fully determined by a finite number of parameters a. The 
Bellman equation for the risk function is replaced by a system 
of equations which specify the evolution of these parameters in 
inverse time. This system is roughly equivalent to the original 
Bellman equation. 

The unit OP (.Figure 1) simulates this system of equations 
and determines the parameters a as a function of time. It 
operates as a self-contained unit, if measurement of the statistics 
of the processes and other variables is not carried out in the 
course of operation, and must finish its work before the start 
of operation of the main system. If the operating conditions 
change, then there may be a need for periodic plotting of the 
process of determination of the parameters by the OP unit in 
application to the new operating conditions. Such a system will 
belong to the class of adaptive systems. The OC unit produces 
the optimal control action in response to the values of the 
sufficient coordinates and the risk function parameters corre¬ 
sponding to a given moment of time. The corresponding algorithm 
is derived from the form of the Bellman equation and the 
adopted approximation of the risk function. 

Usually the transition to a finite number of parameters entails 
some deterioration of the quality of operation of the system. The 
greater the number of parameter taken, the higher the accuracy 
of approximation and the closer the system to optimal, but, on 
the other hand, the more complicated the OP unit. For a 


specified number of parameters is is important to determine the 
successful choice of the means of approximation. Here a great 
deal depends on the ingenuity and inventiveness of the designer. 
In this paper, one natural means of selecting the parameters is 
suggested—taken as the parameters are the bottom coefficients 
of the expansion of the risk function by a suitable full set of 
functions. 

The block diagram of an optimal controller given in the paper 
is of a basic nature, and in fact not all the units need be there. 
In some problems the DK unit can be left out because of triviality. 
The OP unit can be separated from the system. It can be 
replaced by a preliminary calculation, and the parameter 
values can be taken into account once and for all in the syn¬ 
thesis of the OC unit. The situation is different if the system 
itself investigates varying conditions of operation. In that case 
the signals from the appropriate metering devices must be sent 
to the units OP, DK (OC if there is no OP unit). 


Example—Sufficient Coordinates—Stationary Fluctuation Regime 

Let the invariable part of the system—the controlled plant PO 
(Figure i)—have a transfer function K(p). Let the control 
action u be limited to the values - 1 < u < 1. The input signal 
x t , like the output signal y t , is assumed to be known accurately. 
Let the signal on the input x t = s t + £ t be the sum of the pulse 
signal s t — ± 1 and disturbance be the normal white noise 

St (M%t “ 0 ; M£ t £t+x = xd ( T ))- 

The task of the system is to ensure that the coordinate of 
the plant y t reproduces as accurately as possible the pulse 
signal s t . If s t = 1, but y t # 1, the penalty c (1, yd in a unit of 
time is taken. The functions c (± 1 ,yd differ. For the ste P' 
by-step method, which is used to obtain formula (22), the 
condition that these functions be differentiable is essential. 
Henceforward, to make things specific, use will be made of the 
criterion of the minimum mean square error, which corresponds 
to the functions 

c(s,y)=(s-y) 2 (1) 

It will be assumed that the signal s t is a priori a symmetrical 
two-position Markovian process, moreover the a priori prob¬ 
abilities p t (± 1) = P [s t = ± 1] satisfy the equations 

dp(l)__d E i-i) = _ )+ (_i, (2) 

dr dr 


This means that the pulses and intervals are independent 
and distributed according to the exponential law P [t > c] 


It is required to design an optimal controller which produces 
a control signal ut so that the mean penalties are reduced to a 
minimum. The latter is a function of the sufficient coordinates. 

The sufficient coordinates of the given problem will be 
considered. Their definition, which is given by the author 5 ’ 6 
reduces to the requirement of the sufficiency of the selected 
coordinates in three respects: 


(a) Sufficiency 
penalties: 


for determination of the conditional mean 
r t =M[c,\x„u x ,T<i] (3) 


(b) Sufficiency for indication of the constraints of choice of 
the control signal and (c) sufficiency for determination of the 
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future evolution of the sufficient coordinates themselves (for 
the determination of the probabilities of their future values). 

In the given problem the limitations of choice | Uf | <C 1 at 
each moment of time t depend on nothing at all, so point (b) 
can be disregarded. Point (a) will be considered, and the a 
posteriori probabilities w t (± 1) = P [> t = ± i\ x T < t] in _ 
troduced. Then the mean penalities (3) will be written 

r t = c (U y) W t (l) + c(- l, y) w t (-1) 

Requirement (a) will obviously be satisfied if the sufficient 
coordinates include the coordinate y and also the a posteriori 
probability or a magnitude replacing it, say z = w (1)_ w (— 1). 

The evolution of the variables of the given problem will be 
considered. The equation determining the behaviour of y t 
depends on the appearance of the function K(p), Obviously 

^=u ( + n,withiC=l (4) 

and 

dy t 2 dy, 

~df +P ~dt =pUt+pn * (5) 


^ \P) =-r-- 

P +PP 

Assume that n t is normal white noise (Mn f = 0* Mnm 

7 ^ ^ Then in case (4), y t will be (with the fixation of 
{u x )) a Markovian process, and the probability of the future 
values y t + A will be entirely determined by the value at the 
present moment of time. In case (5), the two-dimensional 
process (y b d y t /dt) is Markovian. The probability of the future 
values is determined by these two magnitudes y t , d y t /dt, and 
therefore the sufficient coordinates must necessarily include, 
apart from y h dy t /dt to satisfy of requirement ( c ). If for 
example, disturbance {n t } would be a unidimensional Markovian 
process, then n t should be included among the sufficient co¬ 
ordinates. 

The mode of variation of z t is now found, and it is proved 
that it does not require the introduction of new sufficient 
coordinates. The variation of the a posteriori probabilities is 
induced by two causes—a priori transfers between states s = 1, 

$ 1, and also variation of the a posteriori probabilities as 

a result of supplementary observation of the process x t . If there 
were no observation, the probabilities w* (± 1) would vary in 
accordance with eqns (2): 


= const exp 


Substituting into this 


and relating - 1/2 i 


dw,(l) 




If there were no a priori transfers, the a posteriori probabilities 
(after the observation * t = s + £ t i n the interval r 0 < T <: r) 
could be expressed through the probabilities w 0 (i 1) (before 
this observation) in accordance with the Beiss formula 

w t (s )= const w [{„ t 0 <z< w 0 (s) (7) 

Here w [<?.] is the probability distribution for {£ z , t 0 < t < t] 
which for white noise, as this is known, has the form 


~ L X 7 w **' w *' AVAUVlUg i. I JLt /V j / 

(x t -f 1) d/ to the multiplier C, which does not depend on s, in 
accordance with (7), gives 

w,(s) = Cexp — x t sdz -w 0 (s) 

- K J to J 

From this, differentiation according to t gives 


dw, (s) 
dr 


dC x t s 
C dt + K 


Returning to the case of the a priori transfers, eqns (6) and (8) 
must be combined. This gives 


dw(l) 

~d7~ 


dw(-l) 


==/rw(l) 


= -pw(i)+p W (-i)+ —+F^l W (i) 

_k C d t_ ' 


The derivative 1/C dC/dt is determined from the condition of 
retention of the norm d/d t [w t (1) + Wt (_ i)] = 0 and proves 
equal to - x t /K | [w (1) - w(- 1)]. Substituting this value into 
(9) and transferring to the variable z = w (1) - w (- 1) gives 
the equation 

dz l_ z 2 

dF = — 2/tzd — x, (10) 


which was derived by the author 7 on the basis of the general 
theory. 

Since in (10) x t = s t + £ t , and £ t is white noise, the prob¬ 
abilities of the future values are determined by the value of z. 
and the behaviour of s x ,t > t. But since s x is a Markovian 
process, its behaviour is determined by the value of s h which 
is described by the probabilities w t (s t ), that is to say, once 
again by the coordinate z t . Hence the introduction of new 
variables in accordance with requirement (c) is not necessary. 

Equations (4), (5) and (10) make it possible to write an 
alternative equation or Bellman equation for the given problem. 
Case (4) will be dealt with first. Introducing the function of 
minimum future risks 


S(y,z,t) = minM<\ c r dt\y„z t > (11) 

Ur, U t J 

^J s t ^ le ^ me termination of operation), and compiling the 
difference of these expressions for the two moments t and 
t A, gives the equation 

min M *) + C( | ^ j 


^ (yt>A z t+S(y t ,z ti t) 


+ c\y t9 z t 


(t<T<t + A) 


In computing the limit which stands here, a Taylor expansion 
by the increments y t+A - y h z t+A - z t will be performed and 
both the linear and quadratic terms will be taken into account. 
The differentiability of the risk function is assumed. Eqn (4) 
gives 

U m M^dJZ2l =Ut . A [ mM (ytj^zIl}l = ]s[ (131 

A-+0 A A~>0 A y ; 
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Computation of the Fokker-Planck coefficients for the second 
coordinate z t is somewhat more complicated. In the process, the 

eQUallty M {x t \z t } = M {s t |z,} = z, 

must be taken into account, eqn (10) must be used, and the 
well-known technique of averaging stochastic eqns (10) must be 
applied. The result of the averaging has the form 


is played by the fixation of the risks at the moment of complet¬ 
ing the operation S (y, z, T). If there are no special additional 
considerations, then S (y, z,T) can be made equal to zero. 

The Bellman equation is also derived in a similar way for 
more complex functions K(p). As in case (5), the velocity 
v = by/bt must be included among the sufficient coordinates. 
Then the function S ( y,v 9 z,t ) will satisfy the equation 


lim M 
a-> o 


z t \=-2f,iz t 


+- 


l-z 2 


M z t } + 


(1 “ z t \ 1 

0Z t 


-2 \xz t (14) 


Moreover 


lim M-(y t+A -y t )(y t+A -y t )=0 

A-* 0 n 


lim M 

A-+0 


( Z t + A Z t ) 


Z > = (1 ~^ 2)2 -limM- (Xf+J Xf)2 


(1-2?) 


* A-+0 

2\2 


K 


Hence, eqn (12) adopts the form 


as 
a t 


■+min 


+ • 


as 

0y_ 


. as n a 2 s (i-z 2 ) 2 a 2 s 

' 2/XZ dz + 2 dy 2 + 2 k az 2 


+c(i,n~y~+c(—i,n~2~ = o 


(15) 


(16) 


6 s as as as 

S t +V dy dv ^ dv 


•La- 


dS 


- + 


p 2 N 0 2 S 
2 dv 2 


(i-z 2 ) 2 as 
2tc 0z 2 


+ C (i,j)i4 £ + f ( 




( 20 ) 


An important particular problem among the group of prob¬ 
lems connected with optimal systems synthesis is the problem of 
calculating the optimal stationary mode of operation. In this 
case the operation completing time T tends to infinity. Then, 
irrespective the values of the coordinates at the moment t a 
stationary fluctuation mode is established in the system, char¬ 
acterized by some mean penalty y per unit of time. This means 
that when T increases, e.g., by A t, the risk function increases 
by y At. 

If the difference S(t) - y (T— t) is formed and the limit 
transfer T-> oo performed, the resulting function will not 
depend on time. In case (4) this function 

f(zy)= lim [SO’, 

T-+co 


The second term can also be written in the form — |b5/by|. 
To the resulting eqn (16) must be added the boundary con¬ 
ditions. In view of the fact that \s\ < 1, only the domain \y\ < 1 
need be considered. Because (16) contains the diffusion term 
1/2 Nb 2 S/by 2 on the boundaries y = ±1 there must hold the 
conditions ^ ^ 

4—(±1jZ, 0 = 0 (17) 

ay 

Since 0 < w(s) < 1, for the second coordinate one has |z| < 1. 
On the sides z = ± 1 of the square the diffusion coefficient for 
the second diffusion member 1/2 K (1 — z 2 ) 2 d 2 S/bz 2 vanishes. 
Therefore, instead of the conditions bSjbz = 0 on these sides 
the more trivial conditions 


as can easily be seen in accordance with (16) satisfies the equation 


m 

dy 


+ 2 pz 


dz 


2 dy 2 



z 2 ?U+y 2 -2yz+l~y 


3 Z 


( 21 ) 


[here (1) is used]. Moreover the same conditions (17)—(19) are 
satisfied on the boundaries as before. The solution of eqn (21) 
makes it possible to find simultaneously the function /(y,z), 
the switching line T and the stationary mean penalty y. The 
same holds for eqn (20). 


Solving the Bellman Equation 


as 

0Z 


(y, ±1,0 


< oo 


(18) 


are satisfied. 

will be used to denote the domain of the space of the 
sufficient coordinates, where bS/by> 0, and correspondingly 
R _ where bS/by < 0. The boundary T between R+ and R- 
will be termed the switching line or separatrix; it is to the 
finding of this line that the calculation of the OC unit (Figure 1) 
reduces. On this boundary are satisfied the conditions of con¬ 
tinuity of the risk function and its first derivatives b S/by, b S/bz. 
These conditions are a consequence of the diffusion nature of 
eqn (16). From the continuity of the derivative bS/by there 
follows the condition ^ 

on F • (19) 

ay 


Eqn (16) describes the evolution of the risk function with the 
inverse evolution of time. The role of the initial condition for this 


In view of the difficulty of obtaining a precise solution of the 
alternative equation, various approximate methods can be 
developed. Some of them will be illustrated, taking eqns (16) 
and (21) as an example. Of course the methods—for example, 
the method of parameters—permit generalization to other more 
complex cases as, say case (20), but then the complexity 
of the calculations increases markedly. The results obtained 
with the aid of (16) are also approximately valid for case (20), 
when q > 1, i.e., when the inertia of the controlled plant plays 
a small part and can be disregarded. 

In this case, the optimal control action depends on the 
variables y, z, and equals u = 1 in the domain R+ (correspond- 
ingly 9 u — — 1 in R -). Figure 2 shows the approximate location 
of these domains, and of the switching line; the mean transfer 
velocities M dy/dt, M dzjdt are also given. An approximate 
calculation of the switching line was performed in the stationary 
case (eqn 21), by the asymptotic step-by-step method developed 
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zr(y)=y+ 


2w (1-y 2 ) 2 

k l—4ju 2 y 2 


' 1 
-1 




dydz = min 


With the aid of the inverse matrices 

II cj = \\(<Pi, 9eH _1 ; \Wjm\\ = WWp'l'mW 

the solution of system (24) can be written as 

a ij = Y C ic C jm(S, <Pe4> m ) 


(25) 

(26) 


How the equation for the parameters is obtained from the 
alternative equation will now be shown. Let the latter have the 
form 


Figure 2. Space of sufficient coordinates. POQ: separatrix j / POQBC: 
domain R + ; POQ AD: domain R_ 

by the author 9 . For the case N = 0, 2[a < 1, the switching line 
of the first approximation was found to be 


Ot L J 


(27) 


( 22 ) 


The higher approximations have an order of (ju/k) 2 and 
higher. 

The second approximate method of solution, which has a 
wider sphere of application, will be dealt with in greater detail. 
This method is linked with the determination of the parameters 
of the risk function, to which corresponds the unit OP in 
Figure 1, as was stated in the introduction. 

One of the ways of introducing the parameters is the ex¬ 
pansion of the risk function according to some preselected 
suitable system of functions. For the given example these are 
the functions of the variables y and z. Let (p Q (y), ..., cp r ~i(y) 
and yj Q (z), ..., %-i(z) be the selected functions. Then the para¬ 
meters of the risk function will be the coefficients a i§ (t) of the 
expansion 

S(y,z,t)~ Y E ) ■ (23) 

i — 0 j = 0 

Since the above systems of functions are not complete, re¬ 
placement of the risk function by the expression given usually 
entails some errors. To make the coefficients a i5 more exact, any 
criterion is set, e.g., the minimum integral from the square of 
the difference 


Differentiating (26) according to time, and substituting (27) into 
the right-hand side gives 

^=E^c; m (^'[s] J ^ m ) 

e, m 

If the replacement of (23) is performed here, this will give a 
closed system of equations for the parameters 


day 
d t 


= Y C ie C jm(.£ r lY a p« ( Pp'l / <,']’ <Pe<Am) 


(28) 


The example being considered will be utilized to illustrate 
the application of this method. Because of the boundary con¬ 
dition (17) it is convenient to select the functions (pt(y), each of 
which possesses this property dcpjdy (± 1) = 0. For the second 
coordinate z, there is no such condition, so 

7t y 

r = s = 3;<z>o00=</'o G')=l;4>iO')=V 2 sin T ; 

<P 2 O') : = V2 cos ny;\l/ 1 (z) = z;il/ 2 (z) = z 2 
can be written. 

In the given case (<Pi,(p e ) = c ie = b u \ 


IIGMJII = 


( . _ 1 


9 . 15 

1 0 T 


-j- 0 —— 

4 4 

0 

T 0 | 


0 3 0 

-T° T 
4 4 


(29) 


will be required. 

The variation of this expression leads to a system of linear 
equations 

Y a U (Vi> <Pe) (f/> W = (S, (Pe'I'm) (24) 

i> j 

e = 0, ...,r — l;m = 0, 

which permits a iS to be calculated, if S (y, z, t) is known. 

Here is written 


Since the risk function is symmetrical S(y,z, t ) = S(—y,—z, t) 
(with symmetrical penalties S(y,z,T )), then in expansion (23) 
there should be present only symmetrical terms 

s (y, Z, t)~a 00 + a 02 z 2 + a 11 z y f.2 sin y .v 

+ (a 20 + a 22 z z ) %/2c°S7iy (30) 

Moreover, putting a 10 — oca n ; a 22 = (3a n , it is expedient to 
make the substitution 


0 Vi > <Pe) = - 


(S, cp e \l/J-- 


ViVe&r, 


0S _ n 


Q y IV 2 

a lx 


' 1 
”1 J 


S(p e 4 l mdydz 


V2 


7 1 ~ 

z cos — y — 2 (a 4- /?z ) sin ny 


YPtj( a >P)<Pt(y) zi 


(31) 
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where 

Pij ft) == X; 

1 


<7„ m — 


e,m 

1 fl 


4 J - 


z cos — y — 2 (a+z 2 /?) sin ny 


(32) 

(p e (y)z m dydz 


In addition, within the framework of the selected approxi¬ 
mation 


(1-z 2 ) 


2 32 8 2 

35 7 * y 

1 4 


8 . 71 

71 1 


-J COS Tty 


After the above substitutions, eqn (16), where 1 kc(l 9 y) 
(1 + z) + V a c (-1 ,y) (1 — z) = y 2 - 2yz + 1, adopts the form 

I^^ J = -4 ia u | EPu<P.' zJ ' 
u U£ V 7 17 


+ 2 fi [2 CtQ2 z2 + ^1 ^ 2 ] 

-Z 2 )[a 0 2 + fl2292] 


1/32 8 2 


k \35 1 

— [_a ll zcp i +4(« 20 + ^ 22 z )<^ 2 ] 

1 t 272 871 « 

““ -r- + ~2 92 + ~2~ Z< Pl — 1 

5 71 71 

Equating separately the coefficients of the functions (PiZ J 
gives five equations for the da^jdt derivatives. The most im¬ 
portant of these are the three equations 


=— \a ll \p u \-f^,^) + 2fia 11 +^-Nci 11 +~~2 


d a n _ Ti 

■5T“72 

& a 20 


a 20 a 22 


a ll a ll 


8V2 


n , . [ a 20 

d< 11 20 V«ii’ a ii 


32 ^22 ^ 
35 k + 2 


20 -22 1 + 


272 


da 


22 


20 -r—2~ 

8 . 71 ~ 


. O U 72 /«» xr 

+ 4 ~ 7 . — t --^-^22 


71 1 I / a 20 fl 22 

”dT“vi |aiilP 22 w^y , ^“ 22 ' 7 * 1 2 (33) 

The switching line is found by equating to zero the derivative 
(31). The equation of this line has the form 


4 (a + pz r) sin — y = z r ; z r =z r (y) 


(34) 


The course of the switching line is determined only by the 
relations ix = ~\ ft = —■ of the parameters entering into (33). 

fl 1 1 fl 1 1 > 

As this is usual in dynamic programming, eqn (33) must be 
solved for the inverse passage of time. If the inverse time 
t x = T— t is introduced, the conditions corresponding to the 
end of operation will look like ‘initial’ conditions. In the absence 
of conclusive penalties at the moment T the corresponding con¬ 
ditions will be null: 

a ll = <J 2O = a 22 = 0 when h=o(a = 2;P = 0 


When a sufficiently long time t elapses, the mode of operation 
of the system approaches the stationary. This corresponds to the 
approach by the parameters a ll9 a 20 , a 22 of the stationary values 
a \ l9 ag 0 , a 22 . The latter are the solution of the system of three 
equations obtained by equating to zero expressions (33). 

Using (29) and (34), formulae (32) can be brought to the form 

Pn(x, P) = 3<?nl 

9 15 

Pio(a>jS)= 4-ffio- -jcr i2 ; 


, „ 45 15 

Pi2 (*> P)=J-0i2-J-ViO> 




zf +2 n J 1 -4 +1 




1 — Zr + 3 


j + 3 


sm Tty 


(pi(y) &y 


(35) 


For further calculation of the functions (&,/?) numerical 
methods can be employed, or use can be made of one or another 
approximation of the function z r (y). 

The solution of the given problem consists in the fact that 
the unit OP (.Figure 1) achieves eqns (33) in inverse time, and 
unit OC achieves the switching line (34). 


References 

1 Bellman, R. Dynamic Programming. 1960. Moscow; Foreign 

Languages Publishing House 

2 Bellman, R. and Kalaba, R. Dynamic programming and feedback 

control. Automatic and Remote Control. Vol. 1, p. 460. 1961. 
London; Butterworths 

3 Feldbaum, A. A. The theory of dual control, I-IV. Automat. 

Telemech, 21 (1960), 9, 11; 22 (1961), 1, 2 

1 Stratonovich, R. L. Conditional Markovian processes in prob¬ 
lems of mathematical statistics and dynamic programming. Dokl. 
Akad. Nauk. SSSR 140 (1961) 

5 Stratonovich, R. L. On optimal control theory. Sufficient co¬ 
ordinates. Automat. Telemech. 23 (1962), 7 

« Stratonovich, R. L. Conditional Markovian processes in prob¬ 
lems of mathematical statistics, dynamic programming and games 
theory. 4th All-Union Math. Congr ., Leningrad (1961) 

7 Stratonovich, R. L. Conditional Markovian processes. Probabil¬ 

ity Theory and Its Application , 5 (1960) 

8 Stratonovich, R. L., and Shmalgauzen, V. I. Some stationary 

problems of dynamic programming. Izv. Akad. Nauk SSSR , 
Otdel. Tekh. Nauk. Energ. Automat. 5 (1962) 

9 Stratonovich, R. L. On the optimal control theory. An asymp¬ 

totic method of solving the diffusion alternative equation. 
Automat. Telemech. 23 (1962), 11 

10 Stratonovich, R. L. Selected problems of the theory of fluctua¬ 

tions in radio engineering. Sov. Radio (1961); Topics in the theory 
of random noise, Vol. 1, 1963, New York—London; Gordon 
and Breach 


357 


A Modified Maximum Principle for Optimum Control 
of a System with Bounded Phase Space Coordinates 

S. S. L. CHANG 


Summary 

In Pontryagin’s maximum principle and in ‘bang-bang’ control, the 
manipulated variable is assumed to have no inertia, so that its position 
changes instantaneously from one value to another. In practice this 
never happens, for instance in aircraft controls the elevators and 
ailerons are limited in speed and displacement. The problem is a 
special case of the more general problem of optimal control in bounded 
phase space. In this paper a method is given whereby the problem is 
treated by a limiting process. In place of the rigid bound, a cost 
function with a multiplied K is introduced for regions beyond the 
boundaries in phase space. It is shown that in the limit of K ap¬ 
proaching infinity, both the added cost and the maximum excursion 
of the optimal path beyond the boundaries approach zero, thus deriv¬ 
ing the optimal control condition. The main results of the paper are 
stated in the form of two theorems. Also considered is the practical 
significance of the results for systems with multiple saturation. 

Sommaire 

Dans le principe du maximum de Pontryagin, ainsi que dans le com¬ 
mand^ «bang-bang», on suppose que le variable de commande n’a 
pas d inertie, de sorte que sa position change instantanement d’une 
valeui a une autre. Cependant dans la pratique ceci n’arrive jamais: 
dans les commandes d’avions, par exemple, les gouvernails d’altitude 
et les ailerons ont des vitesses et des deplacement limites. Ce pro- 
bleme est un cas particulier du probleme plus general du commande 
optimale dans l’espace des phases limite. Dans ce rapport on presente 
une methode traitant le probleme par un procede de limitation. Au 
lieu de la borne rigide, on introduit une fonction de cout avec un 
K multiplie pour les regions au dela des limites dans l’espace des 
phases. On montre qu’a la limite de K tendant vers l’infmi, tant 
le cout ajoute que le depassement maximal des limites par le chemin 
optimal tendant vers zero, ce qui assure la condition de commande 
optimale. On etablit resultats principaux de ce rapport sous la forme 
de deux theoremes. On considere aussi la signification pratique des 
resultats pour de systemes a multiple saturation. 

Zusammenfassung 

Bei Pontryagin’s Maximumprinzip und bei der Zweipunkt-Regelung 
wird vorausgesetzt, dafi die StellgroBe keine Tragheit besitzt, so daB 
i .e Einstellung augenblicklich wechselt. In der Praxis kommt dies 
me vor, so sind z. B. bei der Flugregelung Hohen- und Querruder 
nach Geschwindigkeit und Ausschlag begrenzt. Dieses Problem ist ein 
Sonderfall des allgemeineren Problems der optimalen Regelung im 
begrenzten Phasenraum. In diesem Beitrag wird ein Verfahren ange- 
geben,^ bei dem das Problem durch einen BegrenzungsprozeB behan- 
de t wird. Anstelle einer starren Grenze wird fur einen Bereich auBer- 
halb der Grenzen im Phasenraum eine Kostenfunktion mit Viel- 
fachen von K eingefuhrt. Es wird gezeigt, daB, wenn die Grenze von K 
gegen Unendlich strebt, sowohl die zusatzlichen Kosten als die groB- 
ten Abweichungen vom Optimalweg jenseits der Grenzen sich Null 
nahern. Auf diese Art werden die Bedingungen fur optimale Regelung 
abgeleitet. Die wichtigsten Ergebnisse dieses Beitrags werden in Form 
zweier Theoreme vorgelegt. Es wird auch die praktische Bedeutung 
dieser Ergebnisse fiir Systeme mit mehrfacher Sattigung betrachtet. 


Introduction 

The paper aims at removing one essential but impractical con¬ 
dition in the present optimum control theory. In both Pontry¬ 
agin s maximum principle and the better known ‘bang-bang’ 
control, the manipulated variable or rudder is assumed to be 
inertialess 1 - 5 . Its position can be changed instantly from - a 
to a. Yet this is never true in actual ships and planes. 

The problem can be considered as a special case of a more 
general problem, that of optimal control in bounded phase 
space. For instance, in controlling an aeroplane, the elevators and 
ailerons are limited in both speed and displacement. One way 
to remove the multiple limits on the movements of the controls 
is to consider the velocities of the elevators and ailerons as con¬ 
trols only, and to regard the displacements as phase coordinates 
together with the other dynamical variables of the aeroplane. 
Then the phase coordinates representing the displacements are 
bounded. 

In a previous paper, Gamkrelidze 6 gave a necessary condition 
for optimal control in bounded phase space. However Gam¬ 
krelidze s condition is quite complicated and difficult to apply, 
and it has not been proved sufficient for any type of system. 

A simpler necessary condition is derived in the present paper 
for a more restricted class of problem: the boundary in phase 
space is assumed to be returnable, that is, at every point in the 
boundary there is an allowed control vector which brings the 
state vector back to the interior of X. If the system is linear, and 
the allowed regions of phase variables and controls are convex, 
the condition is then both necessary and sufficient. 

The method used here is also different from that used by the 
earlier investigators in that the problem is treated by a limiting 
process: in place of the rigid bound, a cost function with a 
multiplier K is introduced for regions beyond the boundaries in 
phase space. It is shown that in the limit of K approaching 
infinity, both the added cost and the maximum excursion of the 
optimal path beyond the boundaries approach zero, and the 
condition for optimal control is thus derived. 

The problem and assumptions are stated and the main 
results of the paper are given in the form of two theorems. 
Some practical significances of the results for systems with 
multiple saturation are considered and an outline of the proofs 
of the theorems is given. 


Vector Representation, Magnitude, Distance 

Superscripts are used to denote components of a vector, and 
subscripts are used to denote components of a covariant vector, 
e.g., x = (x 1 , x 2 , ..., x n ), and xj/ — (\j/ l9 \j / 2 ,..., \l/ n ). The vectors 
are printed in bold type. Sometimes the Oth component is also 
included in a vector, and this is shown by underlining, e g 
x = (x°, x\ x 2 , ..., x n ). 
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Subscripts on vectors are used to distinguish different vectors 
of the same kind. For instance x x and x 2 are two points in 
x space. 

The concept of ‘magnitude’ of a vector or ‘distance’ between 
two points does not enter into the problem nor the theorems. 
However, in proving the theorems, it is desirable to have these 
concepts so that closure properties and bounds can be defined 
or calculated. The magnitude of a vector x is denoted as \x\ 
and defined by 

|jc| 2 =(a 1 ) 2 + (x 2 ) 2 + ... +(x n ) 2 

The same is true for covariant vectors. The distance between 
two points x x and x 2 is \x x — x 2 |. The distance between a point 
x ] and a region X x is 

min \x — Xil 

.re X i 


The Problem 


The controlled system is described by a set of first order 
differential equations: 

X l =f l (x 9 u 9 t) (/ = 0,1,2,..., ri) (1) 

where x and u are the two vectors (x\ x 2 , ..., x n )\ and (u\ 
ir . u n ') respectively, and the functions f ' 1 are single valued, 
bounded, differentiable with respect to the x’s with bounded 
first partial derivatives, and are continuous in the w’s on a 
product region X L X U 1 X T lt where X lf U l9 and are regions 
in x and u spaces and an interval of t respectively. The vector 
x represents the dynamical state of the system, u represents the 
available controls, and x° is the return (or cost) function. With 
x(/j) given, the problem is to find a u(/) such that 

u(t)eU (t 1 <t<t 2 ) (2) 

x(t)eX (t x <t<t 2 ) (3) 

and 

R = x° (? 2 ) - x° (ty) = maximum (or minimum) (4) 

where X and U are closed regions in Xy and respectively. 
When both eqns (2) and (3) are satisfied, u(t) is said to be an 
allowed control, and x(t) is said to be an allowed path. Among 
the allowed pairs, the u(t ) and x(t) which satisfy eqn (4) are 
said to be optimal control and optimal path respectively. 

If t 2 is given but x(t 2 ) is unspecified, the problem is said to 
be one with a fixed time interval or free end point. 

If x(t 2 ) is given but r 2 is unspecified, the problem is said to 
be one with a fixed end point. Sometimes the end point f is not 
fixed but is a function of / 2 . The problem is reducible to one with 
a fixed end point by using a new state vector x . 

x'(0sx(0-ff(0 (5) 


If in a problem with fixed end point, f° (x, u , t) — 1, and 
R is to be a minimum, the problem is then a minimal time 

problem. 

The controlled system is called a linear system if P (x, w, t) 
can be expressed as linear functions of x and u . 

f(x,u,t ) = F) (0+ 'y Bi (t) U k + c l (0 

j=i k=l 


(i = 0,1,2,..., n) 


( 6 ) 


The region X is assumed to be bounded by simple surfaces 
on which the condition of returnability is satisfied. To be more 
specific 

(1) A unique normal exists on every boundary point x h of X . 

(2) Let tj (x b ) denote a unit vector in the direction of the 
normal at x b pointing outward. The partial derivatives b^(*)/ 
b x j are uniformly bounded. 

(3) Every point x within a certain distance d x from X is on 
one and only one of the normals i/(x & ). The normal t](x b ) 
therefore can also be identified as tj(x\ where x may be any 
point on i/ within a distance d 1 from the boundary. 

(4) There exists a distance <4 d x > d 2 > 0, and a constant 
a x > 0, such that at every point x within distance d 2 from X 
there are at least some ueU satisfying: 

'Z r h( x )'f i ( x ’ u ’ t ) < ~ a i’ ueU 

i = 1 

The closed region extending outward from the boundary of X 
up to distance d 2 is denoted as X Q . 

(5) Z + 

The optimal control need not be unique, but is assumed to 
be finite in number. Thus degenerate systems in which the return 
function is independent of one or more components of the 
control vector for a finite interval are not under consideration. 

The above are the significant assumptions. There are other 
conditions which are assumed to ensure the closure and limiting 
properties. These conditions are usually met by a physical 
system, and are explicitly stated later together with an outline 
of the proof. 

Results 

The results of this paper can be summarized in two theorems: 
Theorem I—Let u(t) and x(t) for h<t^< t 2 satisfy eqns (1) to 
(3). A necessary condition for u(t) and x(f) to be optimal in the 
sense of eqn (4) is that there exist a finite covariant function 
\j/ (f), and a function £(/) satisfying eqns (8) to (10) in the interval 

t-± t ^ t 2 * 

^+l"^^T = C(0*(*) 0 = 0,1,2,..., n) (8) 

k-0 OX 

where C(t) = 0 if Jt(i) is an interior point of X, and C M > 0 if 
x(f) is a boundary point of X; rj t is the ith component of t], 
and?7 0 — 0. 

X ij/ k f k u, t )=max f fa f k (x, u, t) (9) 

k= 0 ueU k ~° 

\J/ 0 (t)= constant >0 

(\]/q (0=constant < 0 if eqn (4) is to be a minimum) 

Furthermore, the following boundary condition on iK?) ls 
satisfied in a free end point problem: 

l Pi(h) = 0 (i=lj2,...,n) (11) 

Theorem 2-Let u(t) and x(t) for t x <t<t 2 satisfy eqns (1) to 
(3) of a linear system, and X be convex. If the conditions state 
in Theorem 1 are satisfied by some \j/(t) and £(t), then u(t) an 
x(t) are optimal control and path functions in the sense of 
eqn (4). 


359 






S. S. L. CHANG 


Theorem 2 can be strengthened by stating that if U is 
strongly convex, or if U is convex and the controlled system is a 
‘normal system’, u(t) and x(t) are the only optimal control 
and path pair 8 . 

Application to Systems with Multiple Saturation Limits 

The optimal control of a system with multiple saturation 
limits is used here to illustrated a typical application of the two 
theorems. The controlled system is described by eqn (1), but 
instead of eqns (2) and (3) 

luUa, ( 12 ) 

W\<b t (13) 

The problem is reduced to one of bounded phase coordinates 
by considering u as the control vector, and (x, u ) as an enlarged 
state vector of n + n + 1 dimensions. Let the enlarged co¬ 
variant vector be denoted by (\j/, (p) with xj/ corresponding to x 
and ip corresponding to n. Then the new Hamiltonian function is 

H(x,u,u,\p,q>,t) = Y J 'l'i f (*,«,0 + Z <Pj u j (14) 

1 j 

Eqn (8) becomes 

. k=n 3 f k 

*t+ Z ^k~Y=0 (15) 

k = 0 OX 

k = n 0 ffc 

<Pi+'Z'l'kT L r=£(t)ri i (16) 

k = 0 CU 

Eqn (9) becomes 

Z <Pj & = max Z <Pj u J (17) 

J \u J \<,b-> j 


In an interval in which \u*\ < a { (pang interval), eqns (16) and 
(17) give 


% n , Qf k 

<Pi=- Z T 

k=o on 

(18) 

tii = bi sign cpi 

(19) 

In an interval in which \u l \ — ai (bang interval) u i 
eqn (17) can be satisfied only if 

== 0, and 

<Pi = o 

Eqn (16) gives 

k= n Qrh 

£ ^/crr>0 if u l =ai 

k — 0 o u 

(20) 

53* 

1 

J 

"a 

<+H 

o 

Vi 

(21) 


Inequality eqn (21) shows that the optimum u 1 maximizes the 
Hamiltonian function at least locally in a bang interval. 

Summarizing the above, the optimal control is a pang-bang 
system: either iP or u i is at an extreme value. There are different 
pang intervals and bang intervals for different components of u, 
and generally these do not coincide. From any point x in phase 
space, the optimal path to the terminal point may not be unique. 
However a set of optimal paths can be selected so that there is 
one and only one optimal path from each point x. If the system 
is autonomous, the optimal set and the associated optimum 
control rate u are time independent. Therefore u can be ex¬ 


pressed as a function of x. As each component u can take only 
three values, 0 and zbzbi, there are optimum switching bound¬ 
aries for each component u i . 

Eqn (15) is a differential equation for determining \jj. Eqns 
(18) and (20) allow <p to be determined from \]/, and the optimum 
switching boundaries are given by eqn (19). A numerical example 
illustrating these points is given by Chang 8 ’ 10 . 

Proof of the Necessary Condition 

The proof of Theorem 1 is contained in an earlier report 9 , 
and only the assumed conditions and essential steps will be 
given here. Because of condition (3) a distance function z;(x) 
can be defined as 

v(x) = 0, ifxeX 

i=n 

o(x)= Z r h(x)(x i -x i b ) if xeX 0 

i = 1 

where x b is the boundary point of X upon which >y(x) falls. 

Let the problem defined earlier be denoted as P. A con¬ 
structed problem P(K) is defined as follows: 

In eqn (1), the equation for i = 0 is replaced by 

x° =f°(x, u,t) — K [u(x)] 2 (22) 

Eqn (3) is replaced by 

x(t)eX + X 0 , t x <t<t 2 (23) 

The other conditions remain unchanged. 

The following Lemmas are then established: 

Lemma 1— Given jc(f x ) e X, and a K > 0, there is at least 
one optimal path in problem P(K). 

Given the initial point x ( t x ) in the n + 1 dimensional x space, 
let the set of all terminal points x(t 2 ), t 2 > t l9 by an allowed path 
be denoted &(t 2 ), and the set of all points x on the allowed 
paths be denoted Q {t <C t 2 ). The crucial point in proving 
Lemma 1 is to show that sets Q {t < t%) are closed. This property 
can be proved by assuming either (a) or (b) as listed below. 

(a) The admissible control function u{t) is measurable, and 
satisfies a preselected modulus of measurability: there is a finite 
M(e,d) for each e > 0 and 3 > 0, and for each u{t) there is a 
step function g (t) of no more than M steps such that 

\g(t)-u(t)\<e 

on T x excepting a subset of total measure (or length) less than d. 

The alternative (b) is to make assumptions on f(x,u 9 t): 

(b.l) The inverse function (x,z, /) is single valued and 
continuous in all its variables. Thus z =/(*, u, t ) maps U into 
a closed region Z (x, t) in z space and u =/(“D ( x ^ z, t) maps 
Z into U. There is a continuous one to one correspondence 
between zeZ and u e U for all x and t on X{T x . 

(b,2) The region Z (x, t) is convex in z space for all values 
of x and t on X{r x , 

(Z>.3) The function u(t) is measurable, t x < t < t 2 . 

Lemma 2—Given any distance d,Q<d<d 27 2 i sufficiently 
large M(d) can be found such that if K > M(d ), v($)<d for 
every point ^ on an optimal path F 0 (K) of P{K). 

Lemma 2 is readily proved from eqns (22) and (4). 
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Lemma 3 —For sufficiently large K , it is possible to find a M 3 
independent of K such that 


ueU such that f l (§, u,t)=0 


( 30 ) 


K 


v(£)dt<M 2 


(24) 


where £(f) describes an optimal path of P(K). 

Lemma 3 is proved from conditions (2) and (4). 

It follows from Lemma 2 that F 0 (K) stays in the interior of 
X 4- X 0 for sufficiently large K. P(K) is thus reduced to an 
optimal control problem without bounds in phase space. Follow¬ 
ing Pontryagin 4 , eqns (9), (10) and (11) are proved together 
with an equation for the covariant function 


The above theorem can also be generalized to linear systems 
with non-linear return function. 

F 0 (x,u,t)=-g 1 (x,i) + g 2 (u,t) (31) 

where g x (x, t ) is a convex function of x for every t. Eqn (6) then 
holds only for i = 1, 2. n. Eqn (26) becomes 

dt ^o 

=*£ j C (0IJ, (*) [*' (0 -(03+E B k (0 [“*(0- (03 


+ E 'I'j 


df j (x,u,t) 
ax 1 


= 2*fro Kv(x) ni (x) 


(25) 


(i=0,1,2, n) 

Either under assumption (a), or under assumption (b) with 
a further condition that / (x,u,t) is separable, a sequence of K 
can be selected such that all the terms and relations expressed 
in eqns (9), (10), (11) and (25) converge to a limit as K->cc. 
Because of Lemma 3, the return function for P(K) converges 
to the return function for P, and eqn (25) converges to eqn (8). 

Proof that the Necessary Condition is also Sufficient for Linear 
Systems 

Let the set u(t), x(t), and f(t) together satisfy eqns (1) to (3) 

and (8) to (11) with the linear form of/* (x,u, f) given in eqn (6). 
Let u(t) and x(f) be another allowed control and allowed path 
pair. Then it is shown from eqns (1) and (8): 

^S>«(0[*‘(0-*'(03 

=Eao^ (*)[*''«-*‘(03 

i 

+E E (0 (0 [« fc (0 - «* (03 ( 26 ) 

i k 

Note that for the problem with fixed end point, eqn (11) is 
replaced by x(t 2 ) = x{,t 2 ) (27) 

Integrating eqn (26) from t 1 to t. 2 and making use of eqns (11) 
or (27) give 

i/'o[*°(0)-*°(0)3 
= f‘ i EC(0»j.(i)[« , (0-*‘(03^ 


+C*‘ (0 - *‘ (0]| 
e* 1 > 

+ i/r 0 [- gi (x, o + g 2 ("> 0 + gi (*> 0 - S 2 ( u > 03 

On the right-hand side of eqn (32) 

EC(07 i W[^(0-* i (03>0 

i — 1 

e" E 'I'a (0 [«“ (0 - (03+g2 («. 0 - g2 («> 0 > 0 


(32) 


i= 1 k 
i = n 


4- 


ti i 
ti 


(28) 


E E 'A; (0 B'k (0 [“ k (0 - u k (03 d * 

It follows from the convexity of X and eqn (9) that both integrals 
on the right-hand side of eqn (28) are non-negative. Therefore 

xKlx\t 2 )-x\t 2 ) 3>0 (29) 

The optimality of x(t) follows from eqns (10) and (29) 

For the problem with fixed end point, the optimal system 
and the system to be compared generally do not terminate at 
“e s'aKil in order to imegode to .he = ™ 

limit U, an auxiliary condition is necessary: Let ? denote th 
terminating point. It is assumed that there is a 


E 8g *l — [*i(0 - ( f )3 ->Ao [«! («, 0 ■- gi (*. 0] > o 

;=i vXi 

because of the convexity of X, eqn (9) and the convexity of 
t) respectively. Integrating eqn (32) from L to t 2 gives the 
more general version of theorem 2. 

The work upon which this paper was based was sponsored by 
the Office of Scientific Research , Air Research and Development 
Command , Washington 25, D.C., under Grant No. AF-AFOSR- 
62-321. 
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DISCUSSION 


W. De Backer, C.C.R. Euratom Cetis, Ispra I Varese, Italy 

I am very interested in the contribution of Professor Chang, not only 
from the theoretical point of view but especially for the computational 
aspects, with which we are very much concerned in the sections of 
analogue and digital computing of CETIS, the Scientific Data Pro¬ 
cessing Center (EURATOM) in Italy. 

In fact, the theorems on his P (. K ) problem (about which we think 
m terms of penalty problem in connection with the ideas of Courant, 
Moser and Kelley) result in an approximation (for finite K), by implicit 
computing, to the Lagrange multipliers as they were introduced by 
Gamkrelidze. This implicit computing technique, which is essentially 
a feedback approach, may be promising not only for the digital com¬ 
puter, but surely for analogue computers, where the constraint viola¬ 
tions can be kept very small by the high-gain amplifiers. 

I shall be very glad to show Professor Chang the first results we 
had with this technique when applied to a very simple test problem. 
Out future efforts will be concerned with the stability problems of 
this technique and I would appreciate any comments Professor Chang 
might care to give. 

S. S. L. Chang, in reply 

I am very interested in the computational aspects as mentioned by 
Dr. De Backer. In fact it is the essential idea underlying the theoretical 
approach. I believe that his work, when published, will be an interesting 
contribution. 


The question concerning the stability problem can be interpreted 
in two ways. I do not know which interpretation is meant, therefore I 
will answer both. 

(1) Stability of the optimum trajectory as the value of K varies: 
I have proved the existence of a limit of the optimum trajectories only 
for a selected sequence of K as K increases, but not for any value of K 
as K increases. However, only in special problems made up by mathe¬ 
maticians the former may not imply the latter. In most engineering 
problems it is quite safe to assume the equivalence of the two. As K 
increases, the optimum trajectory converges. 

(2) The stability of the hardware which is made according to this 
principle: If u is expressed as a function of the initial condition and 
time, the system may or may not be stable. But if u is expressed as a 
function of the state variables, then the system is very definitely stable 
by Liapunov s theorem. I shall outline the process as follows: for any 
final point or condition there is one or more optimal trajectories 
passing through each point in state space. Therefore, there is one 
or more optimal values of u defined for each point in state space. We 
may select any one among the few and obtain u as a function of x. 
Then a Liapunov function can be defined by: 

'*2 

r(A)= f°(x(t),u(x(ty)dt 

J *a 

where 3t a ~ & (t a ) and i ( t 2 ) is the desired terminal point. 
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Summary 

PontryagiiTs maximum principle is used for computing the opti¬ 
mum control function u(t) for a given plant and a given performance 
criterion. If u(t) is bounded, the control is of the bang-bang type in 
many cases. If u{t) is expressed as the function of the state variables, 
that means, u(t) = sgn/Od), the equation fix 1 ) = 0 determines the 
switching surface in the state space. In general, these surfaces are not 
given by simple analytic functions; in particular, not if the transfer 
function of the plant contains complex poles. If the initial state is 
considered to be given by a disturbance of the desired state, initial 
error and error derivatives are finite; in this case the final state is 
given by error and error derivatives being zero, and the switching 
surface goes through the origin of the error phase space. 

Based on experiences with second-order plants, a systematic at¬ 
tempt has been made to approximate the exact switching surfaces for 
third-order plants. There is an approximation of the surface portion 
close to the origin (the so-called ‘inner’ surface) and an approximation 
of the larger portion of the switching surface which is not close to the 
origin (the ‘outer’ surface). Examples show the use of these surfaces; 
their results are compared to results with exactly optimum switching. 
They agree well. 

The extension to fourth-order systems is indicated. 

Sommaire 

Le principe du maximum de Pontryagin est utilise pour calculer 
la fonction de commande optimale u(t ) pour un systeme donne et pour 
un critere de performance choisi. Si u(t ) est borne, la commande est 
bien souvent du type rclais. Si u(t) est exprim6 en fonction des variables 
de l’espace de phase, e’est-a-dire u(t) = sgn/(;d), Tequalion/(;d)=0 
determine la surface de commande dans l’espace de phase. En general 
ce ne sont pas des fonctions analytiques simples qui determinent ces 
surfaces; elles sont tres compliqudes quand le systeme considere a 
des poles complexes. Si dans l’etat final l’erreur et sa derivee sont 
egales a zero, cctte surface passe par l’origine de Tespace de phase. 

A partir des resultats obtenus pour les systemes du second ordre 
les auteurs ont developpS une thdorie d’approximation des surfaces 
de commande des systemes du troisieme ordre. Us donnent deux ap¬ 
proximations de la surface de commande, Tune pour la zone autour 
de 1’origine des coordonndes de phase (nommee ‘surface int&rieure’), 
l’autre pour la partie de cette surface loin de l’origine (nommee 
«surface exterieure»). Des exemples d’emploi des surfaces approxima¬ 
tes sont donnds; les performances de commande obtenues par cette 
methode sont companies aux performances obtenues en employant 
les surfaces de commande exactes. La concordance est bonne. 

L’extension aux systemes du quatrieme ordre est propose. 

Zusammenfassung 

Das Pontryaginsche Maximumprinzip wird zur Berechnung der opti- 
malen Regelfunktion u(t) fur ein gegebenes System und ein gegebenes 
Gutekriterium benutzt. 1st u{t) beschriinkt, so besitzen solche Rege- 
1 ungen in vielen Fallen Zweipunktverhalten. Stellt man u(t ) als 

* This research is supported by the Air Force Office of Scientific 
Research, Air Research and Development Command under Contract 
AF 49 (638)—513. 


Funktion der Koordinaten des Phasenraumes dar, das heiBt, u(t) = 
sgn / (xQ, so bestimmt die Gleichung / (x*) — 0 die Schaltflache im 
Phasenraum. Im allgemeinen geniigen diese Schaltflachen keiner ein- 
fachen analytischen Funktion, besonders dann, wenn die Obertra- 
gungsfunktion des Systems komplexe Pole aufweist. Fur einen An- 
fangszustand, der sich aus Storungen vom gewiinschten Zustand er- 
gibt, sind die Abweichung und die Ableitungen der Anfangswerte 
endlich; in diesem Fall verschwinden Fehler und seine Ableitungen, 
und die Schaltflache geht durch den Koordinatenursprung des Phasen¬ 
raumes. 

Auf Grand der Erfahrungen mit Systemen zweiter Ordnung wurde 
ein systematischer Versuch unternommen, eine Naherung fur die 
exakten Schaltflachen von Systemen dritter Ordnung zu finden. Es er- 
gab sich eine Naherung in der Umgebung des Koordinatenursprungs 
(«innere» Schaltflache) und eine andere Naherung fur den groBen 
Bereich der Schaltflache, dessen Punkte weiter vom Koordinaten¬ 
ursprung entfernt sind («auBere» Schaltflache). Beispiele zeigen die 
Anwendung dieser angenaherten Schaltflachen. Das Verhalten der 
Regelungen mit den angenaherten und den exakten Schaltflachen wird 
verglichen; es ergibt sich eine gute Ubereinstimmung. 

Auf eine Erweiterung dieser Gedanken auf Systeme vierter Ord¬ 
nung ist hingewiesen. 


With the help of Pontryagin’s maximum principle a designer 
can determine for a given plant a control system which is 
optimum for a given performance criterion. Many papers have 
been devoted to the investigation of zeroing initial disturbances 
in minimum time, because this seemed a desirable performance. 
However, the control of spacecraft has drawn attention to the 
fact that minimum fuel consumption may often be more impor¬ 
tant than minimum settling time. In some cases, the problem of 
control with minimum fuel consumption leads to a bang-bang 
control just as in the minimum time control problem. Whether 
this occurs or not, depends on the type of mechanical or electrical 
power supply 1 . The following will be restricted to linear systems 
and to performance criteria which lead to bang-bang control. 
Let the system be given by 

x—AZ + Bu or x l =f l (x J ,u ) (1) 

A is a constant matrix and B is a constant vector. The control 
function u is bounded 

M<1 (2) 

It is desired to go from 

x ( 0 ) = % 0 t0 ( 3 ) 

with the performance criterion 
r 

g (x l ) df-+ minimum (4) 

o 
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Pontryagin’s maximum principle states that the Hamiltonian 

H = 1'.Pif-gix 1 ) (5) 

must assume at any time an extreme value in order to satisfy 
the performance criterion. This extremum will be reached, if 

u = s g n £ ( Pib t ) = — sgn F (t) (6) 

The functions p t are the solutions of a system which is called 
adjoint to the system considered, 


.9^ = _ y 3/ J 

dx l L dx l Pj 


The initial conditions of the functions p { must be chosen such 
that starting at the point x f is reached in time T. In the mini¬ 
mum time case, T is not given but is made a minimum, in this 
case, g (x l ) - l. Without specialization 5c f can be set 5c f = 0 
The above procedure means that to each point x> belongs 
a set of switching points given by F(t ) = 0; see eqn (6) The 
computation of the coordinates of these switching points for 
given initial conditions of the phase trajectory is sometimes 
tedious. If it is possible to find the geometric locus of all possible 
switching points in a phase space [(n - 1) dimensional surface 
lor an *th order problem], then the task of finding the initial 
conditions for the adjoint functions in a particular case is super¬ 
fluous. For second-order systems without zeros this task has been 
achieved for the minimum time case 2 , several minimum error 
criteria and minimum fuel consumption 1 . In the second-order 
case the locus of the switching points is a curve which separates 
tie phase plane in two halves. The initial u value is apparent and 
the only trouble is the realization of this switching curve as a 
unction of the phase variables. The remarkable fact is that the 
switching curves for minimum time and several error criteria 
are neighbour curves. This fact first observed for the (1/s 2 ) plant 
has been used by the authors to find the mean square error 
switching curves for the (1/s 2 + 1) and (1/s 2 + 2 £ s + 1) plants 
by merely perturbing the minimum time switching curve, and 
watching the change of the magnitude of the integral determining 
the performance. ° 


Quasi-optimal Switching Curves for Second-order Systems 

Linear switching as an approximation of an optimum 
control function is excluded from this paper 4 . 5 . The question 
of a better approximation, then, has to be dealt with. For per¬ 
formance criteria which do not lead to chatter near the origin, 
the control law requires a switching curve near the origin which 
is formed by portions of all possible zero-trajectories. Such a 
requirement can be and has been easily satisfied, e.g. in the 
minimum settling time control of second-order systems with 
pure, imaginary poles. The first example (Figure 1) shows an 
often expressed idea of approximation for a system with the 
transfer function (l/s a + 1)6. This idea can easily be extended 
to systems with (1/s 2 + 2 f s + 1), see Figure 2. In this case the 
linear part of the switching curve is made parallel to the ‘envelope’ 
of the cusps. In both cases one can argue whether much is really 
lost by taking only part of the first cusp and the dashed lines. 



Figure 1. Quasi-optimum switching curve for a (1/s 2 + 1) system 



The fact that there exists a mathematical procedure princi¬ 
pally to compute the optimum control law, does not mean that 
the result of such a computation necessarily enables the designer 
to realize this control. It will first be necessary to weigh the 
advantage of an optimum control with a possibly difficult 
control law against a control with a somewhat simpler switching 
function. Only if the advantage is great will it be decided to 
realize the optimum law. 

The simplest switching function, that is, the linear function, 
is excluded from this consideration. This switching function will, 
in general, lead to chatter at least near the origin of the phase 
space if not also in other regions of the phase space. Optimum 
conti ol can also lead to chatter. The case of the performance 



treated by Fuller 3 , shows this clearly, but in general, chatter 
will be avoided if optimum control is used. In systems with 
linear switching functions the chatter is due to imperfections, 
while in systems with optimum control this chatter occurs in an 
ideal or perfect system and the imperfections of the control 
components would only modify this chatter. 


Figure 2. Quasi-optimum switching curve for a (\/s 2 + 2 f s + 1) system 

Second-order systems with real poles require switching curves 
which can be described by a simple power law. Also, the number 
of switchings required can easily be determined; it is one or none 
(for all those initial points which incidentally lie on a zero 
trajectory). The design problem is simple compared with systems 
with complex or imaginary poles. 

Control of Third-order Systems 

For third-order systems the control with minimum settling 
time is certainly the simplest one. Let the general equation of 
a system (e = error = input-output) first be considered 

e"' + a 2 e" + a i e’ + a 0 e = b 2 u" + b 1 u' + b 0 u (9) 

This system has poles and zeros. Switching occurs on a surface 
m the three-dimensional phase space. If b x = b 2 = 0, one has 
the third-order system without zeros, which will be treated here. 
The case of systems with zeros can be reduced to the case without 
zeros by introducing an additional transformation. This case 
will be treated in a report which will appear in the near future. 
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In the case of three real poles, the division of the phase space 
in two halves does not pose any analytical difficulties. The 
realization of this switching surface may pose an analogue com¬ 
puter problem, but certainly not a digital computer problem. 
Therefore, the immediate concern is with the problem of one 
real and two complex poles. This case has been treated recently in 
two papers 7 * 8 . In these papers iteration procedures were described 
for finding the exact switching curves. 

There is no doubt that the exact switching surface poses a 
difficult design problem, particularly if initial disturbances of any 
larger size would be admitted. (The magnitude of disturbances 
should be measured by the bound on the control variable u. 
For instance if <? 0 , e 0 ' and e 0 " are the initial values of the phase 
variables, the norm || T 0 II > |w| max would mean a large disturb¬ 
ance.) Fortunately the experiences with the second-order system 
can be generalized. 

The system described by the following equation is now 
considered 

( s + y ) ( s 2 + 2 £<jos + co 2 ) e =« = — sgn F, with |£ | < 1 (10) 

This third-order differential equation can be replaced by a 
system of three coupled first-order equations in e , e\ and e". 
This system can then be conveniently transformed to a partially 
uncoupled system by the transformation 




i+£ 

CO 

yv 

CO 


co 2 2 £oo 



( 11 ) 


The constants m\ 9 and m 2 depend on the given initial 
disturbance as mentioned earlier. Their determination causes 
trouble which can be avoided if a switching surface can be found. 

The analytic expression for the exact switching surface is 
not known. However, we can visualize the form of this surface 7 . 
This visualization will help us to replace the exact switching 
surface by a simpler surface. Near the origin of the phase space 
the approximation of the exact switching surface has to be made 
more carefully than at larger distances from the origin. 

The approximation for larger distances from the origin 
is developed first and will be called the ‘outer’ switching surface, 
then the approximation in the neighbourhood of the origin, 
called the ‘inner’ switching surface will be developed. 


The ‘ Outer ’ Switching Surface for Plants 1 js (s 2 + 1) 


In the following a quasi-optimum switching surface for the 
plant with the transfer function 1 /s(s 2 + 1) is first developed 
and later it is shown how the results for this plant can be 
generalized. 

For y = £ = 0 eqn (16) simplifies to 

u — sgn [m* cos (cot + /?*) + m 2 ] (17) 


If T is assumed to be the time for zeroing an initial disturbance, 
one can consider the study of the motion in reverse time r. 


t = T-t 

u — sgn [m* cos (cot — coT /?*) + mf\ 
= sgn [m* cos (cot - J?j*) + m 2 ] 


(IB) 


In the new coordinates the system (10) is described by 
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The exact optimum control function u is given by 

u = sgn(±p l +~P2+P 3 ) 


( 12 ) 


(13) 


The functions p t are the solutions of the adjoint system 
Pi-&Pi + v <°Pz\ 

Pi^CtoPi-ytopA ( 14 ) 

p 3 =P3y > 

Integration of this system yields 

Pi = m Y e" ?t cos (vcot + PP)] 

p 2 =-m 1 e" c ‘sin(vcof+/? 1 ) ' (15) 

p 3 = m 2 e yt I 

with n h , m 2 and Pi as constants of integration. Upon introducing 
these expressions into eqn (13) one obtains 


« = sgn 


— m , e ? “‘ cos (vcot + PP) 

CO 


—— m t e ? “' sin (vcot + /?*) + m 2 e? 


= sgn [m* cos ( vcot +/?*) + e 7 ] (16) 


One can construct a trajectory in reverse time by assuming 
P* * and (mllm 2 ). The switching times are then determined and 
after T sec a point in the phase space will be reached which 
corresponds to the initial disturbance. Since cos (co r - coT- 
^f) = cos [co r - (co 7 + ^f)] is a periodic function, two 
trajectories for which (coii + 0ix) — (co T 2 -f /?x*) —2%n 
will partially coincide and have coinciding switching points in 
the identical portions. It has been shown 7 ’ 8 how to find the 



Figure 3. Projection of the optimum switching curve in the x 1 x 2 plane 
for a (1 ls(s 2 + 1)) system 
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location of these switching points in the phase space (see 
Figure 3). It is obvious that it would be difficult to build-up 
the surface on which all possible switching points for all possible 
m\\m 2 , T are lying, but there is no need to be very exact as 
soon as one is one cusp away from the origin of the phase space. 
Therefore it was tried to approximate the locus of the switching 
points in a rather primitive way. The straight lines on which the 



Figure 4. Projections of lines joining the tips of the cusps in the x 1 x 2 
and x 2 x z planes respectively 

points P 2y P^ ... [Figure 4 (a)] and P l9 P 3 ... [Figure 4 (6)] are 
lying, are considered as representatives of the more complicated 
curve which is the carrier of switching points. If one considers 
the ruled surface built of these lines, one obtains for || * || > 1. 

u = - sgn F 


time. As indicated earlier only the surface formed by the trajec¬ 
tories emanating from the two zero trajectories will be con¬ 
sidered. The portion of this surface, for which the distance of 
the surface points from the origin is smaller than a fixed value, 
will be used. For larger distances the surface given by eqn (19) 
will serve. 



Figure 5. Sketch of the quasi-optimum switching surface for a 
(1/s (s 2 + 1)) system 


The limit for the use of the surface near the origin was first 
assumed to be given by [(x 1 ) 2 + (x 2 ) 2 + (x 3 ) 2 ] < (2) 2 . However, 
it soon turned out that the transition from the 'outer’ surface to 
the inner surface can cause trouble. Since the outer surface is 


= - sgn |2 lx 1 1 x 3 + (sgn x 2 ) [(x 1 ) 2 + (x 2 ) 2 ] 

(x^) 2 _(x 2 ) 2 1 

x arc cos |_ (' x i) 2 + ( x 2 ^j | for (x V) >0 (19 a) 



U = - sgn j 2 lx 1 1 [x 3 - (sgn X 3 ) • k] + (sgn X 2 ) [(x 1 ) 2 + (x 2 ) 2 ] Figure 6. Sketch of the behaviour of F vs time as the x z axis 



for (x x x 3 )<0 


(19 b) 


Figure 5 shows a sketch of the surface which includes the x 3 
axis. On x 3 , however, there do not lie real switching points 
though x 1 = x - = 0 yields F = 0. This is because there is no 
change of sign. The function F behaves as indicated in Figure 6. 


The ‘ Inner ’ Switching Surface 

Mention must now be made of the switching surface close 
to the origin. The visualization is easy. In reverse time the phase 
point leaves the origin on a zero trajectory, each point of which 
can be considered as a switching point. That means, from each 
point of the two zero trajectories a new trajectory is emerging. 
These new trajectories form a surface. Each point of this surface 
can be considered again as a potential switching point, which 
means as a starting point of a new trajectory portion in reverse 


not the exact switching surface, the phase point may pass, let 
us say, a switching from (-f- 1) to (■— 1), just before transition. 
However, this switching may bring the phase point to the (+ 1) 
side of the inner surface. Therefore a contradiction exists and 
a chatter occurs which causes the phase point to be trapped in 
the transition region. This trouble can usually be avoided by 
taking [(x 1 ) 2 -f (x 2 ) 2 + (x 3 ) 2 ] < 1. Because of the flatness of the 
outer surface near the origin, it is even better to use [(x 1 ) 2 + 
(x 2 ) 2 ] < 1 as the transition condition. 

The analytic determination of the switching surface close 
to the origin (the inner’ surface) is somewhat troublesome, even 
if one considers its realization by digital or analogue computer 
equipment. Therefore some simplifications are desired. Two 
possibilities are available. 

First possibility: One replaces the plant transfer function 

(s + y)(s^+2£s+l) by A (20) 


366 



OPTIMUM AND QUASI-OPTIMUM CONTROL OF THIRD- AND FOURTH-ORDER SYSTEMS 


For the latter transfer function, the switching surface is given by 


1 3 / 1 2 

= e x + — e 3 + we 3 e 2 + w I — e 3 + we 2 


T -o 


( 21 ) 


with e L = e, e 2 — e , and e s — e. The factor w is determined as 


.. 1 

i 

' 

£ 

il 

+ 

cP 

e 2+~2 e 3 k 3 l 

>0 

and 



w = — 1 for 

1 , 

e 2 +yejleal 

<0 


(21a) 


From eqn (21) the control function is obtained for the neigh¬ 
bourhood of the origin 


u = — sgn F = 


*sgn 


1 3 / 1 2 
e\ +T* + wg 3 e 2 + w — e 3 + we 


2/ J 


( 22 ) 

with the same rule for the signs. 

The deviations caused by using the model (1 A 3 ) instead of the 
correct plant can be visualized in the following figures. 

In Figure 7 the zero trajectories are given in the e 1 e 2 plane 
for a full third-order system and three possible approximations 
to it. The output of the relay is either (+ 1) or (— 1). This 
determines the scale of the figures. Naturally one can have more 
and less agreement depending on the values of y and £ which 
here are unit and zero respectively. 

In Figure 8 projections of these trajectories into the e 2 e s 
plane are shown. It can be clearly seen that only in a rather 
limited region are the zero trajectories of the approximations 
close to the zero trajectory of the original system. 

In Figure 9 projections of zero trajectories for several other 
third-order systems are shown. If the initial values are not too 
large and only one switching occurs between the start and the 
reaching of the origin of the phase space, these curves will give 
an idea of how good the approximation of the optimum control 


will be, if the complete third-order system is replaced by simpler 
ones. 

The second possibility for control near the origin of the phase 
space is modification of the surface given by eqn (19). One 
applies factor N to the control function which diminishes the 
control effort; that means 

u = — sgn F (23a) 

will be replaced by 

u = —N sgn F (23b) 

with N = /(|| x ||) as indicated in Figure 10. A step width A has 
to be chosen and N t = (i + 1) A for jj x || = / A + fi with 
0 < s < A. In this procedure, one may say, the neighbourhood 
of the original is stretched. Naturally one cannot expect to reach 
the absolute ‘zero’, the final state will be a chatter around the 
origin. Also, one has to count on losing some time by dimin¬ 
ishing the control effort. 



Figure 8. Projection of the zero trajectories in the e 2 e z plane for a full 
third-order system and three approximations to it 




Figure 7. Projection of the zero trajectories in the e x e 2 plane for a full Figure 9. Projection of the zero trajectories for several 

third-order system and three approximations to it third-order systems 
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Figure 10. Sketch of multi-level control behaviour near the origin of the 
phase space 

Results 

The eqns (19 a, b) and ( 22 ) for the switching surface may look 
rather complicated, but they can easily be implemented with the 
help of a modem computer. It is expected that new miniature 
digital computing elements can be used in spacecraft. 

A number of examples have been investigated. These ex¬ 
amples were first constructed in reverse time by employing the 
exact switching points, then it was assumed that the initial 
disturbance was given and that the ‘approximate’ switching sur¬ 
faces given by eqns (19) and ( 22 ) were used to zero this dis¬ 
turbance. One example is given here. 

e? = 30-67; e° 2 = 2-93; e°=-7-07 

True optimum and quasi-optimum switching time were practi¬ 
cally the same. In Figures 11 (a) and (b) projections of the phase 
trajectories are shown. Two other examples are given here 

£? = 13j fxj = 13 

£2 = 13 \ i.e. | Xq= 2 
2j [xq = 15 

The optimum time for this example is r oPl = 31*3, and the 
quasi-optimum time is T Qt0t — 32*4, at which time \x}\ < 0T. 
For the r ext example, with 

£$ > = 6*63 | I xj = 0 
£2 = £3 = 0 \ i.e. | Xq = 0 

J ( Xq = 6-63 

r opt — 8*3 and T q%0t — 8-9 was obtained with \xj\ < OT for the 
latter one. 

Outer Switching Surface for Plants with a More General Transfer 
Function of Third Order 

As indicated earlier, there is still the task of developing a 
quasi-optimum outer switching surface for plants with y # 0 
and C 7 ^ 0. Consider first the case when y = 0 and £ # 0. When 
the analytical expressions for the canonical variables as func¬ 
tions of time are compared, it is recognized that for f # 0 in 
the /zth interval* 

x 2 „ = c„e~cos (vx 3 + <5„) (24a) 

with v = (1 — £ 2 ) 1/2 compared with 

x n = C* cos (vx 3 + §*) (24 b) 

for £ = 0 . xl is similarly changed. Therefore it is indicated that 
eqns (19) should change correspondingly. 

* See eqns 12 (a) and 12 (b) with co = 1 . Replace t by x 3 with the 
help of the third equation = — y x 3 4- u or X 3 = u for y — 0. 



(a) 


Figure 11 (a). Projection of the optimum trajectory in the x l x 2 plane 
for a system with y — 0 , £ — 0. Initial disturbance x l = e» = 2 - 93 . 
x 2 = <? 3 - - 7 * 07 , x 3 = 7-5 a, i.e. e x = x 2 + x 3 = 30-67 



i = - sgn j 2 lx 1 1 x 3 e +c |x3 ’ + (sgn x 2 ) [(x 1 ) 2 + (x 2 ) 2 ] 

V) 2 -(x 2 ) 2 " 


x arc cos 


and 


.(x x ) 2 + (x 2 ) 2 J 


for (x J x 3 )>0 (25 a) 


u = — sgn^x 1 [x 3 —(sgn x 3 )7t]e +c 1x31 

+ (sgn x 2 ) [(x 1 ) 2 + (x 2 ) 2 ] arc cos ^ ^ ^ 

for (x x x 3 )<0 


|_(x 1 ) 2 + (x 2 ) 2 


(25 b) 
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This is a somewhat primitive generalization, because a look 
at the figures in (7) shows that one could apply a much more 
strict analysis. For instance, one could build-up the switching 
surface out of the straight lines which are the arithmetic means 
between the lines S 2 \ ... and *S 3 , ... (see Figure 4f. In 

the limiting case (C 0) this new line would converge into 
the old one. 


The situation becomes more involved, when y ^ 0. In this 
case 


X n —“7 + \ X nQ — 


compared with 


u, 


-ytn 


Xn=U n tn+Ct for 7 = 0 


(26 a) 
(26 b) 


Consider for brevity, the case with C = 0. One sees im¬ 
mediately that the line connecting points 0, P 2 , P 4 , .. P 2n is no 
longer a straight line, in spite of the fact that the projection 
into the x l x 2 plane may still be assumed to be straight. This 
assumption is in the frame of the approximation desribed by 
Flugge-Lotz 7 ’ 8 . It is possible to establish a difference equation 

^ ( P2n)~ x (^2 n - 2 ) Q7) 

x i (P 2 „)-x 2 {P 2n - 2 ) 

which leads to a differential equation for the relation x 3 (x 2 ). 
The quasi-optimum switching surfaces F{x L ,x l ,x i ) = 0 then 
can be found. It can be realized easily with digital equipment. 
In the near future it is intended to increase the number of ex¬ 
amples and to compare more quasi-optimum results obtained 
with the truly optimal results. Also, the results are being ex¬ 
tended to include the general case with finite y and finite f. 


A Simple Fourth-order Problem 

The importance of two-axes satellite control directed atten¬ 
tion to a fourth-order problem, described by 

(.v 2 + 2 Ct oj x s 4- to 2 ) 0 2 + 2 C 2 ^ 2 S + ^ 2 ) e ( s ) = u (s) (28) 


example with co 1 /co 2 = 1/3 serves to acquaint one with the prob¬ 
lem. In this case the coordinates in the zth interval are given by 

x i 1 =-^2 + ^ioCOs(m 1 ^ + 5 ll .) 

c°i 

tl 

X?=—j + R 20 cos (C0 2 1, + § 2i ) 

CO 2 

It is obvious that the control-force influence in the x 1 x 2 plane 
is nine times larger than in the x 8 x 4: plane. Figures 12(a) and (b) 
show an example, which was designed in reverse time. One 
recognizes also that in this case an approximation of the time 
optimal control law is rather easily possible. This example is 
particularly simple in that the ratio (o^(o x is an integer. Ad¬ 
ditional, more general examples are being studied. 



Figure 12 (a). Fourth-order system , projection of the optimum trajectory 
into x 1 x 2 plane 


In orthogonal variables the system is given by four differential 


equations of first order. 

i. — V,«1 — 'Ci(D l 0 
0 0 

o 0 -v 2 0 ) 2 


0 
0 

v 2 co 2 

~C 2 “>2. 




x + 

Vl/®i 

U(°2 


j 2 la> 2 _ 


(29) 


The control torque depends only on one control function; this 
clearly is a simplification for a first attack of the problem. e 
case of two steering functions is also being studie . e omo 
geneous system is decoupled in two systems of secon or er. 
However, the optimal control function depends on both re- 
quencies (example £ = 0) 

u = sgn [C x cos (cOjT + <5t) + C 2 cos (. c °i x +^ 2 )] ( 3 ^) 


where t = T— t denotes again reverse time. Therefore the two 
second-order systems are coupled through the control func- 


tion u. 


A representation of the phase trajectory can only be made 
by tracing projections into the x 1 x 2 and the x x planes, n 



Figure 12 (b). Fourth-order system , projection of the optimum trajectory 
into the x z x* plane 
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DISCUSSION 


M. Hamza, ETH, Zurich , Switzerland 

The work of Professor Flugge-Lotz on optimum control is well known 
and is estimated highly. The present paper is very useful in that it 
presents means of approximating the switching surface obtained for 
optimum control, thus overcoming a major practical disadvantage, 
that is, simulation of a complex surface. 

The results in this paper are given for a system having a fixed 
structure. If the structure is variable this problem becomes much more 
complex since a variable switching surface is required. What approach 
would the authors suggest for realizing a variable switching surface? 
Has the sensitivity of the switching surface to parameter changes been 
investigated ? The authors did not mention anything about the error 
obtained using their method, would Professor Flugge-Lotz please 
comment on this point? 

I. Flugge-Lotz, in reply 

May I answer, firstly, the third question of Dr. Hamza. In the past 
year an extensive investigation of the quality of the proposed approxi¬ 
mations to the optimal switching surface was made (with assistance 
of Mr. Y. Kashiwagi after Dr. Titus had left Stanford University). 
This was done in the usual way: a number of exact time-optimal 
solutions were computed in backward time. They start at the origin 
of the error state space and lead, after a chosen time T, to some state 
with finite error and error derivatives. Then the latter ones were used 


as the initial conditions for the sub-optimal control in forward time. 
In general, the sub-optimal control will not zero any error derivatives 

in finite time because the inner switching surface is also an approxi- 

—> 

mation to the exact one. Therefore times for reaching |/| = A (A — a 
very small number compared to |z/| max ) with optimal controls were 
compared. The following results were obtained: 

For y = 0 and £ — 0, where the approximating surfaces come 
closest to the exact switching surface, the sub-optimal solution is very 
good. For y 9^ 0 and £ ^ 0 it is suggested that the approximate 
surfaces be used only for y < 0-5 and £<0*3. Then the deviation of 
the optimal and sub-optimal time for reaching A can be kept below 20 
per cent. In many cases it will be lower. The choice of the surface on 
which transition of the ‘outer’ to the ‘inner’ switching surface should 
take place influences the results: other surfaces than those mentioned 
in the text have been found to be preferable. 

The first two questions of Dr. Hamza are related to the above- 
mentioned facts. If the structure of the system changes slowly (that 
means y and £ change slowly) one can vary the approximate switching 
functions correspondingly and hope to obtain a good sub-optimal 
solution. A digital programme for realizing this could be established. 
However, how good this solution is (that means how close it comes 
to the exact optimal solution), would require exact optimal solutions 
for such cases to be studied first. I thank Dr. Hamza for his questions 
which gave me an opportunity to report about the later development 
of our work. 
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Summary 


In this paper consideration is given to the system 
equations dx 

--) + u(c,y,t) 




of differential 


where x (t) is an //-dimensional vector, y (t) an //^-dimensional vector, 

'rj ( t ) a certain (in general random) vector function, and c a constant 
vector. It is assumed that a certain trajectory x = in phase space 
is given for 0 < t < T(0 < T< oo). Assuming that certain informa¬ 
tion is received on the variation of r/ (/), is it required to choose a 
vector c (problem A), or a vector function y ( t ) (problem B), or a 
vector c and a function y ( t ) (problem C), such that some solution of 
the system precisely or approximately realizes a motion along the 
trajectory x = 0 ( t ). 

The paper shows the methods of programme control used. 


Sommaire 

Ce rapport prend en consideration le systeme d’equations differen- 
tielles suivant: 

A y 

-T- -f(x,r),t) + U(c,y,t) 

at 

dans lequel x(t) est un vecteur k //-dimensions, y(t) un vecteur a 
///-dimensions, ?;(/) une certaine fonction vectorielle en general de 
caractere aleatoire et c un vecteur constant. On admet qu’une certaine 
trajectoire x = 0(0 est donn£e dans Fespace de phase pour 
l’intervalle de temps 0 < t < T (0 < T < oo). Compte tenu que 
certaines informations sont revues concernant la variation de r){t\ il 
s’agit de determiner le vecteur c (probleme A), ou la fonction vectorielle 
y(t) (probleme B), ou encore c et y(t) simultan&nent (probteme C), en 
sorte que la solution du systeme s’identifie ou du moins se rapproche 
de la trajectoire x = 0(r). Le rapport montre les methodes de com- 
mande a programme k utiliser pour resoudre ce probleme. 


Zusammenfassung 

Der Aufsatz behandelt das System von Differentialgleichungen 

A -v* 

— =/(x,Jj,0 + u(c,y,t) . 
ar 

wobei x(t) ein //-dimensionaler Vektor, y(t) ein / 77 -dimensionaler 
Vektor, rj (t) eine bestimmte (im allgemeinen regellose) Vektorfunktion 
und c ein konstanter Vektor ist. Dabei wird angenommen, dab eine 
bestimmte Trajektorie x = ip(t) im Phasenraum fur 0<t^T 
(0 < T< 00 ) gegeben ist. Angenommen, daB man aus der Variation 
von ?? (t) eine bestimmte Information erhalt, so ist es erforderlich, einen 
Vektor c (Problem A) oder eine Vektorfunktion y (t) (Problem B) Oder 
einen Vektor c und eine Vektorfunktion y(t) (Problem C) so zu 
wahlen, daB eine Losung des Systems genau oder angenahert die 
Bewegung entlang der Trajektorie 0(0 darstellt. Die Arbeit enthalt die 
Prinzipien der verwendeten Programmsteuerung. 


Introduction 

Consideration is given to the system of differential equations 

$fi=f(x,ri,t) + u(c,y,t) (!) 

where x ( t ) is an w-dimensional vector, y (t) an m-dimensional 
vector, rj (t) a certain (in general random) vector function, and c 


a constant vector. It is assumed that a certain trajectory x = 0(0 
in phase space is given for 0 < t < T (0 < T < go). Assuming 
that certain information is received on the variation of 7] ( 0 » 
it is required to choose a vector c (problem A), or a vector 
function y it) (problem B), or a vector c and a function y (0 
(problem C), such that some solution of the system precisely 
or approximately realizes a motion along the trajectory x =0 ( 0 - 
The problem formulated in this manner is a problem of pro¬ 
gramme control. 

Let O in Figure 1 be the plant under control, whose object 
is to achieve a certain given mode of operation x = 0 (/). To 
achieve this, a unit Y is introduced that develops a control it. 
In forming the control, use is made of information on the 
operating conditions 0 (t) to be set up and also on external 
influences rj (t); this information may be received in distorted 
form for many reasons, e.g. delays and inertia in the trans¬ 
mission line C, measurement errors, random errors, etc. 



Figure 1 


If the problem had an accurate solution, then the required 
control would be determined by the relation 

u (c, y ( 0,0 = <A'(f) - / (<A ( 0 . *1 ( 0 s 0 ( 2 ) 

However, in a number of cases the system (2) cannot be solved for 
the control vector c or the control function y ( t ). This may be 
due to the choice of an inadequate number of dimensions for the 
vectors c and y ( t ), or the presence of incomplete or distorted 
information on the external influences rj (t ). It may also happen 
that it is possible only to choose the control from some narrowly 
defined class of functions, such as piecewise-continuous func¬ 
tions, trigonometrical polynomials, functions whose modulus 
has a constant limit, etc. 

Thus the impossibility of solving system (2) accurately may 
lead to the statement of a number of problems of variational 
type. Bellman 1 considered such a problem when he used the 
dynamic programming technique to derive a control function 
y ( t ) so as to make the maximum deviation of the system ( 1 ) 
trajectory from the required one a minimum. 

One can state the problem of finding control functions that 
make a certain integral criterion of control quality a minimum. 
These problems may be solved by making use of the maximum 
principle of Pontryagin 2 ’ 3 , of classical variational principles 4 , 
and of the principle of dynamic programming 5 . 

One can state the problem of minimizing the error with 
which the given trajectory satisfies system (1). If it is a question 
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of a minimum mean-square error, then this statement of the 
problem leads to the simplest problems in the theory of mean- 
square approximations 6 . 

Finally, one can renounce all attempts to minimize the 
deviation, and seek a solution that simply gives a sufficiently 
accurate approximation. Thus, for example, Roytenberg 7 seeks 
a control function from the class of piecewise-continuous 
functions to give coincidence with the system (1) trajectory 
at a finite number of points on it. 

It is observed that a distinction should be made between 
two essentially different cases in solving the problem of realizing 
the given process. In the first case the initial points of the actual 
and the required trajectories coincide; in the second the initial 
condition of the actual process may have any value. Normally 
in the second case the control is formed not only according to 
the magnitude of the disturbance but also according to the 
deviation of the controlled quantity from that required, i.e. in 
this case the control system will include feedback. 

Programme Control and Optimal Systems 

It should be noted that if the optimum principle is satisfied 
in one form or another in solving the problem of realizing a 
motion along a given trajectory, then in a number of cases it 
becomes possible to introduce feedback into the control system, 
which permits one to automatically correct the motion along 
that trajectory. This fact proves conclusively the advantage of 
systems designed on the basis of one or other optimum criterion. 
However, a number of difficulties are met in applying optimum 
criteria to the design of programme control systems and tracking 
systems. First of all, they often lead to designs requiring heavy 
computation or to designs that are technically difficult or 
impossible to execute. In this case, one must give up trying to 
satisfy the optimum principle, and restrict oneself to an approx¬ 
imate solution of the problem, with the aim of getting the best 
quality of fit within the bounds of technical possibility. 

Usually in applying the optimum principle one needs a 
knowledge of the process to be realized over its whole duration. 
But it may happen that only certain statistical characteristics of 
the programmed process are known, or even that nothing is 
known about its future. In the latter case the optimal control 
theory is powerless, and the only reasonable approach is that 
of minimizing the deviation of the velocity vector for the current 
point from the tangent vector to a certain curve of pursuit from 
a given class, perhaps determined by a system of differential 
equations. Thus in this case the problem of minimizing the 
deviation of the given process from that required is replaced by 
the problem of minimizing the difference between two vector 
fields, one of which determines the actual motion of the point 
and the other the required motion along a curve of pursuit. 
It should be noted that an analogous result is obtained if the 
control is chosen to give a maximum rate of decrease of a certain 
Liapunov function set up for the perturbed-motion system. 
The above approach, by introducing feedback, enables the 
deviation of the actual process to be rapidly reduced from that 
required, without the future course of the latter being known. 
It is shown below that an analogous result can be obtained by 
increasing the stability of the basic control circuit. 

If the motion to be realized is known over the whole of its 
duration, then the following is the most natural method of 
solving the programme control problem. In the first stage a 


control that gives the most rapid means of reaching the given 
trajectory is sought, and in the second the control that achieves 
motion along that trajectory 8 is found. 

The mathematical theory of optimal control that exists at 
present is basically a theory of optimal stabilization. This means 
that this theory permits, in the simplest cases, by the introduc¬ 
tions of relay devices into the control system, an increase in the 
system’s closed-circuit stability at zero input signal. In other 
words, the quality of the system’s stability is improved, using 
optimum criteria, irrespective of the nature and type of input 
actions that are processed. Clearly such a system will deal with 
input actions in various manners according to their structure. 
Below is given an example of a servo-system that reacts well 
enough to step-function inputs, and consequently also to any 
slowly varying inputs. However, in order to obtain this good 
quality in the system, the requirement for optimum closed- 
circuit stability had to be abandoned. 

Example 

The control system having the block diagram shown in 
Figure 2 is considered. Here / is the input signal, x the output 
signal, K the amplifier unit, A the unit forming the gain k, which 
in general is variable. The problem is to find the optimum law 
of variation, given the constraint | k | < tc 0 , and the condition 
that the error s decreases in some sense in the fastest way. 



Figure 2 


Since in this case the time for complete elimination of the error 
is infinite from a mathematical point of view, the time for the 
error to fall within a given region surrounding the origin has to 
be discussed. Rapid action of this type will be called relative 
rapid action in distinction to the normal type. 

The case where the plant under control is specified by an 
equation of the second order is considered, i.e. where L (/?) = 
p 2 H- ap + b. In this case the differential equation being examined 
takes the form 

s + ae + be — f + af + bf -/ce (3) 

First consider the case where the external input / is absent, and 
find the law of variation of k that gives relative rapid action. 
According to the results of Yemelyanov and Fedotova 9 , the 
gain should be determined by the formula 

k = Kq sgn 8 (Tp T-1) £ (4) 

where T -1 is the negative root of the equation 

Jl 2 + a/l + fe-7c o = 0 (5) 

Now consider the case where / is a step function, i.e. let 
/ = 0 for t < 0 and / = f 0 for t > 0, assuming that the magni¬ 
tude / 0 of the step cannot be measured. Reserving the freedom 
to choose T\ take the previous switching law given by eqn (4). 
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Clearly if k — /c 0 , eqn (3) after the step has taken place will 

have the form w x A 

s 4 -ci8 + (b + k o) (s - e 2 ) = 0 (6) 

where “ bf Q /(b + /c 0 ). 

Correspondingly, for k = — k 0 one gets 

e + ae + (b — k 0 ) (e — 8i) = 0 (7) 

where q = 6/ 0 /(6 — fc‘ 0 ). 

Assuming that k 0 is large enough a qualitative plot in the 
phase plane can be drawn for each of these equations without 
difficulty. Equation (6) in the phase plane corresponds to a 
family of spirals converging to a focus-type special point 0 2 > 0). 
Equation (7) in the phase plane corresponds to a family of 
integral curves of hyperbolic type, with a ‘saddle’-type special 
point (e l5 0) through which pass two integral straight lines whose 
gradients are the roots of eqn (5). 

Assuming now that the switching law is given by eqn (4), 
the phase diagram shown in Figure 3 is obtained, provided only 
that TX x < — 1, where is the negative root of eqn (5), is 
assumed. If the latter inequality is not satisfied, an obviously 
unsatisfactory result is arrived at, since the switching line ( T ) 


WKm 



Figure 3 

given by the equation T& -F e = 0 will be cut by the integral 
curves over the whole of its length with e < 0, while in our 
case the straight line T is a sliding line everywhere except 
over the segment EF, where E and F are the points of contact 
with the integral curves corresponding to eqns (6) and (7). Thus 
the switching line resulting from the relevant optimum criteria 
will have been deliberately abandoned. If the representative 
point M falls to the left of the line T, then it will slide along 
this line as far as F, follow a curve of hyperbolic type as 
far as the line s - 0, then a spiral as far as the right-hand part 
of line T, where it will again start to slide towards E. On arriving 
at E it will approach the point (s 2 , 0) along a spiral if a > 0, 
while if a < 0 it will start to move along a cycle consisting of the 
segment GE of line T and the segment EHG of the spiral. Thus 
any point in the plane arrives, eventually, either within a 
sufficiently small region about the point (e 2 , 0) or at a limi 
cycle corresponding to some self-oscillatory mode. It should be 
observed that the amplitude of the resulting self-oscillations is 
of the same order as e 2 = bfj(b + k 0 ), and consequent y can 
be made as small as required by increasing K 0 . 

It should be noted that by increasing T the length of the 
segment over which it cuts the integral curves is decreased but 
the speed of sliding along this line is also lessened since, as an 
readily be seen, the sliding law is givenby the relahon*- *„ exp 
f- tIT) Thus proceeding from various quality criteria and 
combining^.peculation with experiment . reasonable ..to for 
the time constant T can be selected. 


Together with R. M. Yeydinov and V. A. Tabueva the 
author has been carrying out analogous investigations for a 
third-order system. Here the main difficulty lies in the problem 
of synthesizing a corresponding optimal system. 

Connection with the Accumulated Disturbance Problem 

Returning now to the problem formulated in the first 
paragraph, as far as the approximation to the final section of 
the trajectory is concerned, our problem is directly related to 
that of Bulgakov 10 on the accumulation of disturbances in a 
dynamic system. 

Introducing the substitution z = x — (t) into the system 

of eqn (1) we transform it into the form 

~ - Z (z, n,t) + r (c, y, i If (t), n (t), t) (8) 

where * 

Z (z, ri,t)=f(z + ilf(t),ri ( t ) ,i)-f(xj/ ( t ), n (t), t) 
r (c, y, xp (t), n (t), 0 =/ (xp ( t), n (0. t)-xj/'(t) + u (c, y, t) = r (r) 

System (8) is a system of equations for perturbed motion, 
the function r (t) determines according to eqn (2) the approxima¬ 
tion error of the programming or control functions, and the 
deviation of the solution z (t) of system (8) from zero coincides 
with the deviation of the solution x it) of system (1) from the 
given function i p (t). 

If system (8) is linear, then for z (0) = 0, 0 < t <T < °o 
we have z (f) = Ar (r), where A is a linear-bounded operator 
transforming the function r (t) into the functions z (t). If |M|| is 
the norm of the operator, then ||z(t)|| <S |M|| Ik (Oil is obtained. 
The latter relation is also the most general expression of the 
solution to the problem of disturbance accumulation. By taking 
various norms for r (0 and z (0 and computing [Mil, the actual 
inequalities that solve this problem 11 - 12 are obtained. 


Connection with the Theory of Approximations 

If as an optimum criterion that of the minimum error r (?) 
(in any dimension) is taken, then the problem of realizing the given 
trajectory reduces to a problem in the theory of approximations. 
This problem is most effectively solved in the case where r it) 
depends linearly on the programming parameters and functions, 
and where we require a minimum of the mean-square approx¬ 
imation error. In this case the elementary rules of the theory of 
mean-square approximations are used for computing the control. 
It should be observed that here two essentially different cases are 
met. In the first case, by selecting the programming parameters 
from a sufficiently large number of the latter the approximation 
error can be made as small as required, i.e. the given motion can 
be achieved as accurately as necessary. In the second case the 
error of approximation cannot be made less than a certain value. 
Here it is worth while to state the problem of simultaneously 
choosing optimal values for the parameters and optimal pro¬ 
gramming functions. The success of such a choice depends, 
roughly speaking, on how well the given ^trajectory fits mto 
linear subspaces in the various dimensions . 

Trajectory Realization and Stability Theory 

If it is wished to approximately realize motion along a given 
trajectory for the whole interval 0 < t < co, certain difficulties 
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arise. It can readily be seen that such an approximate realization 
is possible if the zero solution of the system 

dz 

—=Z(z,ri,t) (9) 

is stable in relation to continuously acting disturbances that are 
limited relatively to the dimension in which the approximation 
error r ( t ) is evaluated. There exist 13 stability criteria related to 
continuously acting disturbances limited in modulus or in 
mean value. Stability criteria can easily be deduced 14 for use 
with continuously acting disturbances limited in their mean 
square, which are of most interest in our problem. However, in 
solving the problem it was required to find convenient evalu¬ 
ations of continuously acting perturbations that were simulta¬ 
neously evaluations of approximation errors. 

Such evaluations 14 were found and it turned out to be best 
to make them in the dimension of space M with norm 

[*(fc+l)T 

||r(«)|| 2 = sup \r(t)\ 2 dt 

0 <k< cc J kT 

where | r (f)| denotes the length of the vector r (t). Massera was 
the first 15 to point out the important role of the space M in 
stability theory. 

Dwelling further on a question related to stability theory, 
the operating mode \j/ (t) is called stable in relation to the 
system x = X (x, t) if the zero solution of the system 

i = x(z + il,(t) 9 t)-xmt) 9 1) 

is asymptotically stable. From the preceding argument it is clear 
that only stable operating modes can claim to give a good 
approximation. Unfortunately few criteria for operating mode 
stability have so far been derived in relation to this system. 
Clearly if the basic system is linear and asymptotically stable, 
then any operating mode will be stable relative to it. The same 
property is possessed by the systems considered by Krasovskiy 
in his paper 16 (theorem 3.1). These systems are determined by 
the fact that for each of them a constant symmetrical matrix A 
can be defined having positive eigenvalues such that the 
symmetrized matrix 



has negative eigenvalues ^ satisfying the inequality < - d, 
where d > 0 at all points of the space — oo < < oo, 0 <t< oo. 

The interesting result obtained by Letov 17 is also noted, 
concerning non-linear control systems with parameters that 
vary only slightly. He has proved for a large class of systems of 
great importance in control engineering that the stability of a 
given operating mode implies the stability of alt sufficiently close 
modes. In this case the closeness of the modes is assessed by the 
magnitude of the modulus of the difference between the pro¬ 
gramming functions. 

Probably further results in this direction can be obtained 
on the basis of both existing and new criteria for asymptotic 
stability of linear systems with variable coefficients. It can easily 
be verified that in the unidimensional case Krasovskiy’s criterion 
is a necessary and sufficient condition for the stability of any 
mode. It would be interesting to know to what extent this 
criterion is necessary for systems of a higher order. 


Realization of Periodic Motions 

Now let the right-hand side of the system (1) and also the 
function ijj (/) be periodic in t with period T, Assuming that the 
zero solution of system (9) is asymptotically stable to a first 
approximation, we can again formulate the conditions for a 
given motion to be realizable with the required accuracy. But in 
this case these conditions can be set more simply, since here 
the dimension in space M is given by 

IK0ll 2 = Jo \r(t)\ 2 dt 

Furthermore it can be shown that even in the presence of an 
approximation error different from zero there exists an asymp¬ 
totically stable periodic motion lying within an e neighbourhood 
of the given periodic motion. 

It should be observed that the results obtained can be ex¬ 
tended without difficulty to the case where the motion to be 
realized is discontinuous, or more accurately has discontinuities 
of the first sort 14 . In this case the programming functions will 
appear as the sums of ordinary functions and linear combi¬ 
nations of <5 functions. 


Programme Control of Random Processes 

Up to now attention has not been directed to the external 
influence or, more precisely, disturbance r\ (/). Normally rj (t) 
is a random function, and so the actual mode of operation will 
be a random process. Naturally in this event the programmed 
mode is also random. The extension of the preceding results to 
the case of stochastic differential equations presents no difficul¬ 
ties, provided the following points are borne in mind. A random 
quantity, as is known, may be determined as a measurable 
function defined in some choice space 2 (or space of elementary 
events). It is easy to see that the space 2 can be constructed in 
such a way that it is the choice space for all random functions 
V (0> £ (0 and x ( t) occurring in the equation 

dx 

—=/(x,t,t](t)) + u(t,^(t)) ( 10 ) 

where £ ( t) is the distortion of the disturbance rj (t) (see Figure J). 

If a norm is defined by any means in the linear space of 
random quantities (as in the space of measurable functions 
defined in the choice space ±), then differential eqn (10) is 
transformed into a differential equation given in the linear 
normalized space R, whose elements are random vectors. Here 
one should take as initial vectors in the solution of Cauchy’s 
problem not only deterministic vectors but also any other random 
vectors from R, while the derivative and integral of a random 
function w.r.t. t should be understood as the derivative and 
integral in Bochner’s sense. In particular, if the mathematical 
expectation of the square of the length of the vector is taken 
as the square of the norm of a random vector, then the concept 
of the derivative and integral of a random function coincides 
with the generally accepted one. 

It should be observed that the theory of differential equations 
in a Banach space is well developed at the present day. By 
making use of this theory, one can readily formulate conditions 
for the existence, uniqueness and extensibility of solutions 18 , 
and consider questions of stability 19 or questions of the ex- 
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stence and research of periodic motions 20 . All this enables the 
setting up of a completely analogous statement of the problem 
of realizing random processes and to obtain results identical 
to those presented above 21 . 

The reduction or elimination of the effect of disturbance by 
its continuous tracking has found wide application in the 
theory of automatic control, mainly in the theory of composite 
control systems. This theory uses the so-called invariance principle 
developed by Academicians Luzin and Kulebakin, which has 
served as the starting point for a large number of papers on 
automatic control theory that have important applications. 

Realization of Processes by means of Systems with Many-valued 
Characteristics 

Barbashin and Alimov 22 have shown how to reduce systems 
of differential equations with relay-type hysteresis, and in 
general many-valued characteristics to a differential equation 
in a normalized linear space. Thus in this case also all the 
preceding results can be obtained by the same method as was 
indicated for the programming of random processes. 

Conclusions 

It has been seen in this paper that the accuracy of approxima¬ 
tion to the trajectory depends on the degree of stability of the 
zero solution of the system (10). The better this stability, as 
judged according to any of the existing quality criteria, the 
smaller effect will approximation errors have on the deviation of 
the trajectory from the given one. Thus the problem of improving 
the response of programme control turns on the problem of 
increasing the stability of motion. Here, in particular, the 
theory of programme control again comes into contact with the 
theory of optimal control. 
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DISCUSSION 


I. Flugge-Lotz, Engineering Mechanics Division , Stanford University , 
Stanford , California , U.S.A. 

The author mentions the work by Yemelyanov and Fedotova, which 
promises a good follow-up of slowly timed varying inputs. The switch¬ 
ing law is non-linear. However, since sgn [e (Tp + 1) e]I - sgn £ sgn 
nrp + 1) e it can be realized easily. In 1957 Charles F. Taylor and I 
developed a control system which allows a good follow-up of inputs 
with strong slope discontinuities (sawtooth curves). The system is given 
by the following equations where/ (t) is a given input. 

/'+2C(l+/fJ/+(i+?0W(0 


with ]? m =iiSsgn(y'e)-2/5sgn(yV) 

y„ = - sgn (ye) - 2 ? sgn (ye') 

Extensive testing of that system is reported in NACA Tech. Rep.1391 
(1958) Since Yemelyanov’s and Fedotova’s switching law is so 
attractively simple, I would be interested in knowing whether ex¬ 
periments have been conducted to show the follow-up and its linn- 

tations. „ ,. . 

As to third-order systems, I assume that the difficulty lies m a 
reasonably accurate subdivision of the three-dimensional space in 
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regions in which the system is governed by linear differential equations 
with desirable parameters. Did the authors consider using digital 
control ? 

Ye. A. Barbashin, in reply 

The example Professor Fliigge-Lotz has reported shows that the 
optimum system with respect to the zero input may prove useless for 
an uneven action. This fact is well known by experts of the theory 
of automatic control. Examples like that might obviously be given 
in great numbers. In order to give an example, I resorted to the paper 
by Yemelyanov and Fedotova, which presents the optimum solution 
of the problem. 


This paper does not deal with systems of variable structure. These 
are discussed in the paper by Petrov, Ulanov and Yemelyanov. It 
seems therefore to be more appropriate to refer to Mr. Yemelyanov 
who is investigating the system of variable structure. Since the field 
of mathematics deals also with these systems, I refer to the papers 
published in the journal Automatika i Telemekhanika 10, (1963) Nos. 1, 
5, 7. In Nos. 1 and 7 will be found the results of the experiments 
concerning the method suggested. Yemelyanov’s paper clearly shows 
the advantage of similar systems. 

As to the application of discrete computing devices for the system 
of variable structure, this problem has been theoretically developed and 
is now in the stage of experimentation. 
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The Realization of Optimal Programmes 
in Control Systems 

G.S. POSPELOV 


Summary 

Optimum control problems are, as a rule, a priori programmes which 
are formed either by mathematical programming methods (maximum 
principle, dynamic programming, etc.), or by control theory methods. 
A mathematical model is required, but since this model does not 
coincide with the actual process, additional devices and feedbacks are 
required for optimalization. Several examples are given which illustrate 
how optimum programmes can be formed. 

Sommaire 

Les systemes d’optimalisation sont en general des programmes a 
priori derives, so it de methodes mathematiques (principe du maximum 
programmation dynamique, etc.), soit de methodes deduites de la 
theoric de comma nde. Un modele mathematique du systeme est 
necessaire. Cependant puisque ce modele ne coincide pas avec la 
realite du processus a optimiser, des contre-reaclions et des dispositifs 
supplementaircs sont ndcessaires pour l’optimalisation. Differents 
exemples sont donnds qui illustrent comment des programmes d’opti¬ 
malisation peuvent etre 6tablis. 

Zusammenfassung 

Probleme der optimalen Regelung fiihren im allgemeinen zu „a priori- 
Programmen“, die entweder durch mathematische Programmier- 
methoden (Maxinuimprinzip, dynamische Programmierung usw.) 
oder durch Verfahrcn der Regelungstheorie gebildet werden. Man 
braucht ein mathematisches Modell, aber da dieses Modell nicht mit 
der wirklichen Strecke iibercinstimmt, werden zur Optimalisierung 
weitere Geratc und Ruckfuhrungen benotigt. Fiir die Aufstellung von 
Optimalprogrammen werden einige Beispiele angegeben. 


Methods of mathematical programming [the term is used to 
mean the application of mathematics to the practical activity 
of planning, development, decision-making etc., and is a 
natural generalization of such concepts as linear (or non¬ 
linear) dynamic programming] are spreading to all branches 
of the national economy, engineering, industry, agriculture and 
so on. This presupposes the development of mathematical 
models of the events or sets of controlled plants which require 
to be controlled. Once the aim of control has been formulated, 
the task is to determine the optimum strategy of control or 
a programme of effects upon the controlled plants produces in 

some sense the optimal result. 

It must be emphasized that the programming methods 
determine the strategy of control or a priori programme. The 
degree of coincidence between the actual result or process pro¬ 
duced by control and the result or process anticipated from the 
a priori programme, is indicative, in particular, of the perfection 
of the mathematical model or of our knowledge of the 

controlled plant. , 

However, a mathematical model is a model and not the 
phenomenon itself, and, apart from this, during the process of 


realizing the a priori programme, the controlled plant can be 
affected by a variety of factors and perturbations which are not 
taken into account in the model. This can lead to deviations, 
and sometimes to considerable deviations, from the programme 
results, which by definition are optimal. 

If the programme is time scheduled, use can be made of 
feedback to correct the effect of perturbations and inaccuracies 
in the mathematical description so as to ensure an actual 
programme closer to the optimal one. 

Those most completely represented by mathematical models 
are control systems. Taking their case as an example, we will 
consider the possible ways of realizing optimal programmes, in 
this instance, control programmes. 

A mathematical model of a control system is usually formed 
by means of ordinary differential equations. The control pro¬ 
gramme is broadly defined to cover the planning of the dynamic 
characteristics of the control system, its programme of operation, 
etc. In all cases it is assumed that the system is provided with 
complementary feedbacks which improve the realization of the 
predetermined programme or a priori programme. 

(1) The desired dynamic characteristics of a system are 
realized by complementary self-adjusting circuits, which in this 
case are complementary feedbacks which improve the realization 
of the predetermined programme of the control system of 
operation. Figure 1 shows the well-known self-adjusting system 
of an automatic pilot which controls the angle of pitch of an 
aircraft 1 . The self-adjustment circuit changes the gain of the 
angular velocity circuit so that the margin of stability of this 
circuit is maintained constant. The correcting circuit 2 is selected 
to obtain a sufficiently high gain K. Under these conditions the 
transfer function of the closed angular velocity circuit is close 
to unity. Therefore, despite the variation of the properties of 
the controlled plant (owing to changes in flying conditions), the 



Figure 1. ft-angle of pitch; ft programme -value of pitch angle; 1-con- 

trolled plant; 2 - correcting circuit; 3 - model; 4,5- measuring devices 
for the angular velocity ft and the angle ft; 6,7- detectors; 8,9- high 
and low pass filters; 10 - servo motor; 11 - limiter 
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dynamic properties of the angle of pitch circuit will be deter¬ 
mined by the transfer function of the model, i.e. in all cases 
they will be quite close to the predetermined or planned prop¬ 
erties. Another example is the self-adjusting control system with 
extremal tuning of the correcting circuits 2 . Both examples refer 
to continuously operating control systems. 

A somewhat special problem arises in the preservation of 
planned dynamic properties for ‘single-action’ systems 3 for which 
the behaviour is significant on a finite interval t (0 < t < t x ), 
and for which the operating process is, as a rule, a transient 
process. Here one meets with the problem of maintaining a 
desired nature of transient behaviour, or a programme of 
motion of the representative point in phase space, on condition 
that the mathematical model does not exactly describe the 
dynamic properties of the controlled plant, nor the perturbations 
acting on the latter during the motion. Several possible ways 
of solving this problem are now indicated with simple examples. 

Let the mathematical model of the controlled plant be 
represented in the form 

x = u (1) 

where x is the output coordinate and u is the controlling action. 
Given the equation of the controller under the form 

u=-a 0 x (2) 

the equation of the mathematical model of the system as a 
whole is 

x + tf 0 x = 0 (3) 

According to (3), for any initial condition x 0 , the process of 
motion is characterized by an exponential with the exponent — a. 
Now suppose that there is a suspicion that, in fact, the control 
object is described by the equation 

x = f{x,u,t) (4) 

where 

f(x,u,t)=-a(t)'(/)(x) + F{t) + u ( 5 ) 

a (4 F(t) are random functions and cj) (x) is also random. 
Here it is known beforehand that | a (t) (f> (x) -+• F(/)[ < | u |. 
In this situation one can make a decision concerning the discrete 
control of the plant, such that at each step it is possible to con¬ 
trol the fulfilment of the a priori programme, which is expressed 
as a function of time in the following manner: 

x=x 0 e~ af (6) 

With discrete control we require for control a relationship 
between the value of x ( t) and the value of this coordinate at the 
instant of time t + At, i.e. the quantity x (t + At). According 
to (6) this programme relationship is given by the relation 

x(t + At)=x(t)'e~~ aAt (7) 

where At is the interval of discreteness or the step of control. 
Using an analogy between the numerical solution of differential 
equations by difference methods and the discrete control of 
controlled plants, one writes the equation (4) in discrete form 



The discrete form (8) of the solution of eqn (4) is used in the 
method proposed by Bashkirov. (The method of Bashkirov is 
described in the monograph by Popov 4 .) According to eqn (8), 
by measuring the value of x (/) at each step one can select the 
increment A u at the instant t + (A//2) such that x (t + At) is 
governed by condition (7). The discrete form (8) is convenient 
in that the interval At/2 is given in the procedure for calculat¬ 
ing A u (t + At/2). The information for calculating A u (t + A//2), 
apart from the known value of the desired x (t + At), is obtained 
from the preceding values of A u and x. In the general case 
Au (t + At/2) is calculated by the formula: 


Aw( * + y) = Au^-y J-l/r 


At 


x(i+Af),x( f-y ),x(t-At) 


(9) 

The form of the function yj depends on the particular theory of 
extrapolation which is adopted. 

The information about the preceding values of x and u also 
includes information about changes in the properties of the 
plant and of the perturbation F(t). The use of this information 
for calculating Au (t + At/2) represents the additional feedback 
signals, or self-adjusting signals, and makes it possible to realize 
more accurately the desired programme of motion 6 . 

Equation (4) and its results can be generalized without 
difficulty to multi-dimensional systems of any order. In this 
case the equation of the controlled plant in the vector form is 


—=f(X,U 9 t) (10) 

where X is the vector with the components x t (z = 1, 2,..., n ), 
/is the vector with the components/ (/ = 1 , 2, ..., n), and U is 
the control vector with the components zz* (z = 1, 2,..., y); 
y < n. 

The achievement of planned dynamic properties of single¬ 
action systems can also be realized by a continuous control. 
Suppose, for example, that the mathematical model of the 
controlled plant is written in the form 


x + a 1 x = u (11) 

and | u | < u 0 . 

Suppose also that it is required to realize the system with 
maximum operating speed. According to Pontryagin’s maximum 
principle 5 , the equation of the controller is of the form 

u ~~ u m sign [x+f (a l5 x)] (12) 

However, there is a suspicion that in fact the controlled plant 
can be described by the equation 

x + af(tyx + a*(t)-x = u+F(t) (13) 

In view of the incomplete information about a t * (; t ), a 0 * (t) and 
F{t) it is impossible to accept the control law of type (12) 
which would ensure the maximum operating speed. 

In view of this one proceeds as follows, forming the accelera¬ 
tion control circuit x = n by means of the controlling action u 
(Figure 2). If the pass band of this circuit is sufficiently high the 
error s n = n vr — n will be close to zero and the programme 
acceleration will be equal to the actual acceleration. In more 
complex cases the acceleration control circuit, like the pitch 
angle control circuit {Figure 1 ), can be a self-adjusting circuit. 
If now the programme acceleration is close to the actual accelera- 
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tion, any desired variation of the coordinate x and its derivative 
may be required. Thus, to form the system of maximum operat¬ 
ing speed in accordance with the mathematical model (11), it is 
sufficient to put 

x pr £ i : = - a x x - u ! sign [x + f (a l9 x)] (14) 

The block diagram which realizes (14) is shown in Figure 3. 
In expression (14) u ± is always less than u 0 since some part of the 
control resource u 0 — u x goes to compensate the perturbation 



£n = n pr -n P = dt 


Figure 2 


Calculator for x pr 



Figure 3 


F(t) and to compensate the difference between the coefficients 
aj* ( t ) and a Q * (t) on the one hand and the coefficients of the 
mathematical model a x and a 0 = 0 on the other. Thus, at the 
expense of some reduction of operating speed (since u x < w 0 ) 
a definite realization of the programme for the optimum 
transient process is obtained. 

Any other law of variation of the coordinate x can be 
required in this example. It may, for example, be required that 
the transient process should take place in accordance with the 
solution of a linear equation with constant coefficients 

xFa 1 x + a 0 x = 0 (15) 

For this, it is obviously necessary to put x vr = — a x x — a 0 x. 


Figure 4 shows oscillograms which have been obtained on 
the electronic simulator for the case when | a 0 * (01 <0-05; 

| aj* (t )1 < 1-0; a x = 0-4; a 0 = 0*04. The gain of the servo 
motor of the acceleration control circuit was taken as 10 1/sec. 
It will be seen from the oscillogram that the perturbation F(t) 
and the fluctuations of the coefficients a x * (t) and a Q * (t ) have 
no effect on the course of the coordinate x which is governed by 
the solution of eqn (15). 

The results explained by this example are also capable of 
very wide generalization. The generalization consists in that 
for a known indeterminacy of the properties of the controlled 
plant and of the acting perturbations it is advisable to organize 
a self-adjusting subsystem of rapidly varying coordinates of the 
controlled plant or of its higher-order derivatives. After the 
programme variation of the rapidly varying coordinates or of 
their higher-order derivatives has been largely determined by 
this subsystem, the law governing the variation of the slowly 
varying coordinates or lower-order derivatives of the output 
magnitude of the controlled plant can be built as desired. The 
additional feedbacks which make it possible to realize the 
required programme of dynamic properties of the system are, 
in the example under consideration, the feedbacks amongst 
which are the self-adjusting circuits for acceleration control. 

Very often the realization of desired dynamic properties for 
single-acting systems is handicapped by unfavourable combina¬ 
tions of initial conditions. In non-linear systems these unfavour¬ 
able combinations of initial conditions can lead to instability 
of the process for a given realization. The effect of unfavourable 
combinations of initial conditions can be eliminated by changing 
the initial values of the coordinates and by the formation of special 
signals which act on the system and which are functions of the 
initial conditions. Briefly, this means creating special feedbacks 
with respect to the initial conditions. The idea of using feedback 
with respect to the initial conditions has already been published 
in a paper by the author 6 . 

(2) In developing systems with programme control of the 
output coordinates of the controlled plant use may, to a large 
extent, be made of the foregoing ideas and methods which relate 
to the realization of programmed dynamic properties of control 
systems. 

Suppose, for example, that it is required to vary according 
to the programme g vr (t) the output coordinate x(t) of the con¬ 
trolled plant (Figure 5). For this the input of a closed system 
consisting of the controlled plant and the controller receives the 
programme signal g pr (t). For a system with a high pass band, 
if no perturbations are present, it is well known that x = g vr (t). 
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However, a random perturbation which is not taken into account 
can considerably distort the desired programmed variation of 
g vr (t). In order to fulfil more accurately the programme, an 
additional feedback is formed (shown by the dotted line in 
Figure 5) and the programme correction circuit abcdega is there¬ 
by formed. The programme signal g m (t) is compared with the 


aircraft, as calculated, for instance, by the method of dynamic 
programming, is a programme in the coordinates H and V 9 
i.e. it is given as a functional relationship H w = H w (V$ r ) 
CFigure 6), both the quantities H and V here being the output 
coordinates of an aircraft controlled by the altitude rudder (the 
thrust of the engine is usually maximum in this case). The 



Figure 5. ~ controller; W 0 - plant; W k - self-adjusting correction 

circuit with a high gain 



actual signal and the difference signal acts at the input to the 
fundamental system via a self-adjusting correction circuit with 
a high gain Wk . The correction circuit may consist of the 
elements 2, K, 6, 7, 8, 9,10 and 11 which are shown in Figure 1. 
Assuming, for the sake of simplicity, W k = K , the following 
operator relationship is obtained between the input and output 
for the circuit of Figure 5: 


Hprft) 


Auto-pibt Aircraft 




Figure 7 


KrtO+jq 

i - K<p ip) lrW+ i+K^(py (,) 

where 

. , w W p w 0 w 0 

^ 1+ W p W 0 an (t>f(p ^i+w p w 0 

and expression (16) can be written as 


(16) 


X = - 


<Kp)[y+ 1 


+4>(p) 


g P M+Y L f( - — .F F ( t ) (i7) 


■+4>(p) 


It will be seen from (17) that if K-> oo 

X = g(t)pr (IB) 

independently of the action of the perturbation F{t) and the 
fluctuations of the parameters of the controlled plant. It is 
understood that in this case condition (18) is fulfilled approxi¬ 
mately since K = co is not realizable in actual conditions. 

Another example of programme control is the method of 
stabilizing acceleration {Figures 2 and 3) with subsequent con¬ 
struction of the desired programmed variation of the coordinate 
x 0 by means of a computer. 

Using this method the logarithmic navigation’ 7 can be 
realized when the acceleration according to the programme 
k x/x, and consequently, the coordinate x is the solution of the 
differential equation 

xx—kx=Q 


A very important case of programme control is that when it is 
important to maintain a functional relationship between one 
coordinate and another. For example, the optimum programme, 
as regards operating speed, for the altitude and speed of an 



Programme 

mechanism 


Figure 8 

relationship H w — H vr {V pr ) can always be represented para¬ 
metrically : 

H pr =H pr (t) 

V pr = V pr (t) 


The altitude control circuit H can now be formed by the usual 
method {Figure 7). If the system is unaffected by perturbations 
and the calculated characteristics of the aircraft coincide with 
the actual characteristics, and if the atmosphere through which 
the aircraft is flying remains standard, the completion of the 
programme {t) will at the same time imply the completion 
of the programme V w {t), and consequently of the programme 
relationship H w — H m {V pr ). However, if all the stated condi¬ 
tions are not fulfilled, the completion of H w {t) will not generally 
imply the fulfilment of V vr {t\ and consequently the completion 
of H w = H w {V). For the planned programme H vr = { V ) 

to be fulfilled with acceptable accuracy, it is necessary to intro¬ 
duce a programme correction circuit 8 . For this purpose the 
programme value of speed is compared with the actual speed 
and the difference in terms of the transfer function W lc changes 
the rate at which the programme is delivered, i.e. the speed of 
the clocks of the programme mechanisms H vr and V pr {Figure 8). 
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As a result the speed of the clock mechanism of the programme 
is not uniform and the programmes H w and V vr become func¬ 
tions of some irregularly varying argument r, i.e. H pr (t) and 
Vj> r (t). Elimination of the argument r again brings us back to 
the original relationship However, in so far as the rate 

of delivery of the programme signal at the input of the system 
conforms to the fulfilment of the speed programme, the accuracy 
of the realization of H w = H w ( V pr ) is substantially increased. 
A similar circuit can be constructed for the motion of some 
controlled plant along a prescribed unperturbed trajectory 
y e = y e (X e ) in the coordinates x, y (Figure 9). However, this 
report is confined to the plane problem. Suppose that the speed 
of the plant is V and that the orientation of the speed vector 



is characterized by the angle ip. The obvious relationship be¬ 
tween the coordinates x, y and the speed is expressed as follows 

y = V simp+W y (19) 

x = V cos \j/ + W x (20) 


where W v and W x are perturbations in the form of speeds of 
displacement of the environment relative to the system of co¬ 
ordinates x,y. [In the formulae (19) and (20) the actual values 
of the coordinates of the controlled plant are used. The values 
of the desired unperturbed trajectory are denoted as x c and y e .] 
Consider the kinematic problem, i.e. suppose that the angle ip 
of the speed vector can be arranged arbitrarily. On this assump¬ 
tion the control circuit for the coordinate y is formed. Here it 
is required that 

sin \l/ — k E s (21) 

where 

£=y P r=y ( 2T > 

the term y„ = » (t) here being the programme value of the 
coordinate y, which does not coincide, as will be seen below, 
with the unperturbed value y e = y e (*)■ 

The equation for the coordinate y is found from the equa¬ 
tions (19), (21) and (22): 

y + Vk c y = Vk e jv(0+ W y (23) 

Assuming y = y e + Ay, we obtain now the equation for the 
deviation Ay from the unperturbed motion . 

Ay+Vk e Ay=Vk e (y pr -y e )-y e +W y (24) 


It is worthwhile to select the programme signal y vr in accordance 
with the formula 


y pr =ye+ 



(24a) 


For this value of the programme signal, eqn (24) becomes 

Ay + Vk c Ay = W y (25) 

This implies that in the absence of the action W y the deviation 
from the unperturbed trajectory will tend to zero. A constant 
action will cause a constant error. 

It is obvious that a single control circuit according to the co¬ 
ordinate y cannot ensure the necessary control of the coordi¬ 
nate x or the fulfilment of the required programme y e = y e (x 6 ). 
As the coordinate x varies according to the expression 


x=V 


J tan ijj e • Ay dt + J 


cosi/^df—F tani/vAj>df+ W x dt (26) 


The first term in eqn (26) is the desired unperturbed value of 
* = x B9 the second term can be limited, since it is determined 
by the error in the circuit for the stabilization of y, and the third 
term for W x = const will continuously increase. In order to 
realize the programme of motion along the unperturbed trajec¬ 
tory it is necessary to proceed in the same way as in the previous 
case (see Figure S), i.e. it is necessary to form, by measuring the 
error x vr — x, a signal which acts on the speed of the programme 
mechanism y vr (t) and x pr (r). 

It should be noted that it is much simpler to correct the 
programme by varying the speed of the programme clocks if 
in the first example dV/dt > 0, and in the second example if 
dx/d t > 0. Generalizing, this method of correction to the pro¬ 
gramme of a system with n coordinates and r controlling devices, 
we shall note that in this case the argument of control (the non¬ 
decreasing coordinate V in the first example, and the non¬ 
decreasing coordinate x in the second) should be any constant 
sign form of system derivative 9 . 

Frequently this form of coordinate originates naturally from 
the statement of the problem. For example, this is the case if it 
is required to control the ingredients of a mixture as a function 
of the volume of this mixture when this volume is varying in a 
monotonous way. 
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DISCUSSION 


R. Petrovic, Institute for Automation and Telecommunications, 
Bircaninova 20, Belgrade, Yugoslavia 

It seems that the problem of deviations from the a priori optimal 
process dynamics due to small perturbations entering the process in 
some uncontrolled fashion, can be successfully investigated using 
the adjoint method and adjoint computational technique. 

Small uncontrolled perturbations along the optimal state trajectory 
and/or along the optimal control trajectory can result either in a 
deviation from the desired process final state or in a change of the 
process performance index. To establish the dependence between the 
small perturbations along the optimal trajectories in X and U spaces, 
and the final state deviations or the functional changes, it is assumed 
that the real dynamics of the process are close to the a priori optimal 
dynamics. In this way the ‘additional dynamics’ along the optimal 
dynamics are the solution of a linear vector differential equation, 
in a matrix form 

d/d* .x 0 (t\ x (tf u (t) = A (t) x 0 (if x (0, u (0 (1) 

(*0 (0 is the index of optimality as a function of time; [A(t)] is 
(1 -f /z -f y) • (1 + w + y) matrix of coefficients whose elements are 
the first partial derivatives of functions f 0 ,f x ... f n u x ...u y with respect 
to the state and control corordinates). 

To the homogeneous linear vector differential equation (1) cor¬ 
responds the adjoint vector differential equation 


d/dr [> (r)] =-[A (r)] r |>( /)] (2) 

Since the matrix product between the transposed adjoint solution and 
the solution of the original set is a scalar independent of time, one can 
obtain the relation between the solution families of the original set 
of eqn (1) at a fixed time t*e ( t 0 , r t ) and the solution of the adjoint 
set in time running backwards from the real time, started at / = /*. 
Thus, in order to investigate the effects of all the small state and/or 
control perturbations on the deviation from the a priori final state 
it is sufficient to achieve the response of the adjoint system ip (r), 
T = h — U by applying in a proper order the initial values to the 
components ip x ... \p n . Similarly, to analyse the sensitivity of the 
process performance index it is sufficient to find the response of the 
adjoint system ip (r) by applying an initial value to the component ip 0 . 

It should be pointed out that to find the analytical solutions of 
the adjoint set is generally impractical, but, from the computer 
mechanization point of view, it can be easily performed by means of 
models with time-varying coefficients 1 ’ 2 . 
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Some Bounds on Quantization Errors in Dynamic 
Programming Computations 

J. J. G. GUIGNABODET 


Summary 

The application of the Principle of Optimality leads very naturally to 
a computational procedure for the solution of optimal control prob¬ 
lems. In the numerical treatment, however, as a result of the use of 
various interpolation schemes, errors are generated that propagate 
throughout the computation in a cumulative manner. 

In this paper we formulate the basic computational problem and 
calculate upper bounds of these errors in the case where the state of 
the process under study is quantized and also in the case where, in 
addition, the search of the extremal is carried out over a finite set of 
discrete values of the control variables. 

Sommaire 

L’application du principe d’optimalite conduit tres naturellement a 
une methode de calcul permettant de resoudre des problemes de com- 
mande optimale. La solution numeriquecependant, a cause de Futilisa- 
tion repetee de certaines formules d’interpolation est entach£e d er- 
reurs qui se propagent tout en s’accumulant. 

Dans cette note nous formulons le processus fondamental de calcul 
et nous ddterminons la majoration des erreurs dans le cas oil l’etat du 
systeme (Studie est quantifie et aussi dans le cas ou, en plus, Textremum 
est determine sur un ensemble fini de valeurs discretes de la variable 
de commande. 

Zusammenfassung 

Die Anwendung des Optimierungsprinzipes fiihrt ganz naturlich auf 
Berechnungsverfahren zur Losung von Problemen der optimalen 
Regelungen. Die zahlenmaBige Behandlung ist allerdings wegen der 
Yerwendung verschiedener Interpolationsmethoden mit Fehlern be- 
haftet, die sich durch die ganze Rechnung in gesteigertem MaBe fort- 
pflanzen. 

In diesem Beitrag werden die grundlegenden Probleme der Berech- 
nung formuliert und eine obere Fehlergrenze, sowohl fur den Fall, 
daB der jeweilige Zustand des betrachteten Prozesses quantisiert wird, 
als auch fur den Fall, daB auBerdem der Extremwert durch eine be- 
grenzte Anzahl diskreter Werte der RegelgroBe gesucht wird, aus- 
gerechnet. 

A number of control problems may be reduced to finding an 
optimal sequence of decisions {«&} which maximizes the func¬ 
tional N 

h N (x)=max £ F (x„, u„ _ x ) 

{uk} 1 

subject to constraint equations of the form 

X n+1 = 8 ( X n> ®n) 

u k eU 

where g is the process transition function, x n the state at the n th 
decision stage, and XJ a closed region in the control or decision 
space. Such a problem may be treated by the functional equation 


technique of ‘dynamic programming’ 1 . However, because of the 
nature of the constraints, an analytical solution is generally not 
obtainable and one must resort to computational procedures 2 . 

Consider the case where both x and u are scalars and 
assume that interest is in the values of hjsripc) for x in a closed 
interval Xn. Indeed, hjsf(x ) can be evaluated only at a finite sub¬ 
set of discrete values in that interval. Let X* be the set of all 
even multiples of X, 

X* = {0, +2A, +4A, ...}anddefineX*==X* nX N , 

The functional equation formulation of the problem is now 
h n (;x*) = max [F (g (x* 5 u),u)+h n ^ 1 (g (x *, u))] 

u 

with the additional constraint 

It is clear that if /^(A*) is to be evaluated in this way for all 
x*eX$, then %-iO*) must be evaluated for all x* e Xn -i 
where 

t =X* n X N . u X N - ! = ixZ xf ] 
x“ (xf) = min (max) g (x, u) 

ue U 
xeX N 

Thus, h n (x*) must be evaluated for all x* e X* with 
X: = X*nX„, X„ = K_„,x^_„] 

x^_ n (xj!f_„)= min (max)g(x,u) 

ue U 
xeXn+i 

The computation goes as follows: 

h t (x*)=max F(g (x*, u), u ) 

U 

is evaluated first at all x* G X* and tabulated, then one proceeds 
with h 2 (x*), 

h 2 (x*)=max [F (g (x*, u), u)+h x (g (x*, u))] 

U 

with x*eX$. Since, in general, g(x*, u) £ X* an approximate 
value h 2 {x*) is obtained by replacing hi {gix^'^ii)) by hi(g(x*,u) 
4- /) in such a way that 

(g(x*,u) + l)eX* 
leA , A-(-a, X) 

where the interval A is open on the left and closed on the right 
(or vice versa) to insure the uniqueness of l for all x* and u. 

This technique is then applied throughout the computational 
procedure by means of the equation 


383 




J. J. G. GUIGNABODET 


K 0*)=max [F (g(x*, u),u)+h c n ^ 1 {g (x*, u) + /)], 

u 

n- 2,3 ,N 

Let X = \J n X n , it will be shown that if F{x,u) and g(x s u) 
satisfy Lipschitz conditions over X 

| F(x',u)-F (x\ u)\<K\x' — xT 

\g(x',u)-g(x",u)\<K’ \x'-x"\ b 

ueU 

then 

lim h c n (x*) = h n (x*) 

and an upper bound J n (T) of the error | h n (x*) — h c n (x*)\ will be 
calculated. 



In order to get an upper bound as small as possible, the 
variations of F and g will also be bounded over each set X n 
separately. 

Namely, 

\F(x',u)-F (x" ; u)\ <K n \x'- x"\ a (1) 

\g(x',u)-g (x", u)| < K' n \x' - x"\ b 

x',x"eX n , ueU 

Clearly, K n < K and K n ' < K' 

J n (X) is arrived at in the following way: 

Since 

max | fix, y) + h (x, j>)| < max |/(x, j;)| + max | h (x, y)| 
and 

|max f(x , y ) - max h (x, y)\ < max |/(x, y) - h (x, y )| 
y y y 

Then, with the constraint (g(x*, u) + /) e X^~ i, 
max \h n (x*)-V n (x*)\ 

X* e Xn* 

< max lh„- 1 (g(x*,u))-h n . 1 (g(x*,u) + l) 

ueU 
l e A 
x* 6 X n * 

+ K-i(g(x*,u) + l)-h c n ~ 1 (g (x*, u) + T)\ 


<J n - 1 (X)+ max |/i„_ 1 (g(x*,w))-/i„_ 1 (g(x*,w) + /)| 

u eU 
l e A 
**eX n * 

{X)=j n (X) 

where is obtained in the following way, with (g(x*,ii) 

-f /) e Xn~\ 

max |?i„- 1 (g(x*,u))-/2„_ 1 (g (**, w) + 01 

ueU 
l e A 

x*eX* n - t 

< max {|F (g [g(x*, u), u'],u’) — F (g [g (x*, u) + 1 ,«'], u')\ 

u,u' e U 
l e A 
^eX„* 

+1 - 2 (i [g 0*, «)= u']) + h n _ 2 (g [g (x*, u)+1, u'])|} 

Since 

g [g (x*, u), m'] , g [g (x*, u) + 1, u'] e X„_ 2 

it follows that the upper bound of the first term is K n - 2 (Kh^\X) a . 
In the second term, the upper bound of the difference between 
the arguments of the functions /z w _ 2 is K' n _\ X h ; therefore, one has 
the recursive relation 

J'n-l(X) = J'n-2 (K - ! k*) + K n _ 2 (K' n ^ , V*) 

Now 

fti (x*) = hl(x*) i.e., J 1 (A) = 0 
and, with x* e X* 

l-FCg [gO*, u ),«'], u’) — F(g [g(x*, u) +1, u’l u’)\ 

< K 0 K'“X ab = J[ (A) 

The result can thus be formulated as 

Theorem I 

If the functions F and g satisfy the Lipschitz conditions (1) 
and if the interval of quantization of the state variable x is 22, 
then an upper bound J n (T) of the error \h n {x*) — h£(x*)\ is 

Jn(X) = J n -1 W + J'n-1 (fy , n>2 

J'n -1 00 =J ’„-2 (K-X) + K n -2 (K a - l X tb ), n> 3 

(A)=o, r 1 (X)=K 0 K' 1 a r b 

Explicitly, 

J„ (1)= r b (K 0 K'°+K 1 K- 2 a +...+K n . 2 K'^ 1 ) 

+ X abl (K 0 K?K' 2 ab + ...+K n ^K' n a _ 2 K' n ab 0 

i -jab 3 /tv' r^fajy'/abr^-rab 2 , , p- iv'/a r / rrab 1 \ 

+ A [K Q ig 1 K 2 A 3 + ... + K n „ 4 .K n _ 3 K n _ 2 K n _ 1 ) 

+X abn ~ l K 0 K' 1 a K' 2 ° b ... K'gY 2 , n>2 

Clearly, 

J„(X)<K{(n- 1) X ab K' a + {n- 2) (1 + 6) +... 

_j_^aZ> n - 1 j^ / a(l+& + b 2 + ... + & n - 2 )| 
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If a = b = 1, this later relation becomes simply 

AK[(n-l)K' + (n-2)X' 2 + ...+X m_1 ] 

Since a , b , and K are positive, it is seen that J n (2) goes to 
zero uniformly with 2. 

In general, however, both x and u are quantized. Let U* be 
the intersection of U and the set of all even multiples of \j/. The 
search of a maximum over the continuous range U is now re¬ 
placed by the search of a maximum over a finite set U *. One 
can write 

h n (x*)=max [F (g(x*, u * + p), u*+ p) + ft,- i (g(x*. u *+ p»] 

U* + p 

where u* e U* and p e T, ¥ = (- i//, f) i.e., the search over the 
continuous range U is done by trying out successive discrete 
values u* and letting p vary continuously over the range ¥. Now 

h c n (**)=max [F (g (x*, u*), «*) + K-i(g (**» u *)+ 0] 

U* 

and 

\F(x',u')-F(x"y)\<K n \x'-x"\ a +K\u'-u"\ c (2) 

| g(x,u‘)-g (x, u ")I < K”' \u' - u"| j 
x',x"eXj u',u"eU 

Following the same approach as before, an upper bound /„(A,i/0 
of the error |/; n (x*) - A"(x*)| is derived. Setting^ = ^OA^+A, 
the result is 


Theorem II 

If the functions F and g satisfy Lipschitz conditions (1) and 
(2) and if the intervals of quantization of the state variable x 
and the control variable u are respectively 22 and 2\j/> an upper 
bound Jni^xj/) of the error | h n (x*) - f&(x*)\ is 

J„ (A, (p„) +K- 1 4/ C +K n - iJCT"» 

J^tA) = K'M c + K 0 K"[ a r b , J'M = K 0 K[ a p 2 ai 

Explicitly, 

J„ (A, iP) = r(K'o + K'[ + ...+ K- 1) 

+ (K 0 KT+KiK% a +...+K u -iK a ) 

+ K 0 KTpf + K.K'oyf +... + X„_ 2 JC_ ipf 

+ KoK'fKf... K' n a T 2 3 K a -i Vf” ‘ 1 , n > 2 

The extension of those two results to the case where x and « 
are vectors is straightforward. 
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DISCUSSION 


Further Remarks by the Author 

When applied to a practical problem, Theorems 1 and 2 rapidly give 
unreasonably large values of the predicted error bound as n increases, 
from which one can draw few conclusions about the true behaviour 
of the computation errors. In fact, numerical experimentation has 
shown that a practical envelope of the errors is always a function of n 
with a decreasing derivative and thus does not increase nearly as 
rapidly as J n (2) and J n (2, vO. This is due to the fact that, at some 
stage of the computation, only a fraction of the largest error at that 
stage contributes to the largest error at the next stage. This is intuitively 
obvious and can be given a mathematical interpretation by introducing 
the coefficients oc and p smaller than one, as shown below. With 
a = b = 1 and X n = X for all n, Theorem 1 can be rewritten 

J n (2) = aJ„_ 1 (A) + J M _i (A), n>2 
J n - 1 (X)=pj n - 2 (K f X) + KK'L , n > 3 
j 1 (2)=0, J'i (X)=KK'X 

Indeed, we no longer have a true upper bound but a practical way of 
choosing the quantization interval of the state and control variables, 
the size of the set X n at which h n (x) needs to be evaluated, and, 
eventually, the sampling intervals so as to carry out the computation, 
keeping the errors within prescribed bounds. The coefficients oc and p 
can be determined from numerical experimentation for large classes 
of computational processes. 


R. Boudarel, Centre FEtude et de Recherches en Automatisme, 3 
Boulevard Victor , Paris XV e , France 

In his very interesting paper devoted to an important problem of error 
computation, the author considers the case in which the theoretical 
recurrent equations are replaced by approximations such as 

h c „ (x *)=max F (g (x*, u), u) + K _, Q (g (x*. h)) 

u 

where Q (g) is the quantization operator 

Does the evaluation of h^-i (g (x* 9 u)) by a simple (or a higher 
order) extrapolation, in spite of frequent discontinuities of functions 
h n (x) t yield more important errors than evaluation by simple round¬ 
off? 

J. Guignabodet, in reply 

I wish to thank Dr. Boudarel for pointing out the very important 
problem of interpolation procedures. The ‘cost 5 or ‘return 5 functions 
h n (x) that are encountered in control processes, are usually continuous, 
and increasing the order of the interpolation surfaces would certainly 
improve to a great extent the accuracy of the computation. However, 
such procedure would require a very large number of matrix inversions 
and cannot be considered in practical cases. The use of orthogonal 
polynomials to approximate h n (x) over the entire range of interesting 
values of x seems quite promising whenever polynomial interpolation 
is itself meaningful. This procedure must be used with great care and 
the simplest interpolation formulae are often the most satisfactory. 
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THEORY OF SELF-ADJUSTING SYSTEMS 


Adaptive Control 

A Survey by JOHN G. TRUXAL 


In the few years since the founding of the International Feder¬ 
ation of Automatic Control, and indeed in merely the three 
years which have elapsed since the First International Congress 
in Moscow, the confraternity of control engineers has demon¬ 
strated adaptive characteristics far surpassing the most elegant 
of man-made control systems in speed of response, brevity of 
settling time, and insensitivity to external noise and changes of 
personnel. For in that brief period of less than a decade, control 
engineering has developed from a narrow interest in linear, 
single-loop electromechanical controllers to a breadth of concern 
which encompasses the themes of modern stability theory, 
optimization techniques, adaptive systems, finite automata, 
learning and pattern recognition, and which touches upon such 
diverse scientific fields as man’s concern with natural physical 
phenomena on earth or elsewhere, with man himself and the 
operation of his neural and physiological mechanisms, and with 
the analysis of man’s own social institutions. As a result of this 
adaptivity of the control engineering species, this Second I.F.A.C. 
Congress convenes on a non-nationalistic basis with a common 
desire to exchange information on recent advances in the control 
engineer’s quest for improved technological tools to move ahead 
in this multiple role in furthering the conquest of space and the 
understanding of physical, biological, sociological, and economic 
phenomena. 

In this rapid growth of modern control engineering, a central 
theme has been that of adaptive control: in non-mathematical 
terms, the control system in which the dynamic characteristics 
of the controller are purposely designed to vary in accordance 
with unpredictable variations of the state variables. Here the 
state variables are to be interpreted as including input signals, 
environmental variables, and those process variables which in¬ 
dicate the values of process parameters. Furthermore, in the 
evaluation of the adaptive system, there should be a penalty as¬ 
sociated with the failure of the controller to react in accordance 
with the changes in the state variables. 

Such a loose definition of adaptive control clearly encom¬ 
passes a great many conventional systems as well as systems which 
are designed and analysed as adaptive. For example, from an 
appropriate viewpoint, the familiar single-loop system with a 
saturating forward amplifier is adaptive, since the equivalent 
gain of the non-linearity depends upon the error signal level 
(and hence the input signal); furthermore, the quantitative 


evaluation of system performance certainly depends upon this 
non-linearity. In such a case, however, no obvious purpose is 
served by terming such a system adaptive; we avoid inclusion 
of such trivial cases by imposing the requirement that the con¬ 
troller be purposely designed to be insensitive to the variations 
of the state variables: in other words, that the system be 
designed from an adaptive viewpoint. 

While the literature of control theory is replete with argu¬ 
ments about the definition, progress in adaptive control techno¬ 
logy has not been impeded by failure of the purists to reach uni¬ 
versal agreement on an appropriate definition. On the contrary, 
the three years since the First I.F.A.C. Congress have wit¬ 
nessed fundamental developments in two quite separate direc¬ 
tions : 

(1) The development of a fundamental theory of adaptive 
control. 

(2) The design of practical adaptive controllers. 

As a result of these two distinct directions of research and de¬ 
velopment, important contributions have been made in four 
primary areas: 

(a) Control theory. 

(b) Control engineering. 

(c) Control education. 

( d ) Control in related fields. 

In the following, we outline the nature of the developments in 
(1) and (2), and then attempt to depict the nature of the contri¬ 
butions ( a)-(d ). 

Theory of Adaptive Control 

In spite of the surge of interest in adaptive control, approxi¬ 
mately six years have produced disconcertingly little fundamental 
theory. Part of the source of the difficulty can be easily demon¬ 
strated. Figure 1 shows the terminology of the fundamental 
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Figure 1. Control system notation 
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Figure 2. Basic adaptive system 


control problem: here v is the vector representing the observable 
process outputs, u the control signal, n the disturbances, and r 
the input signals. The fundamental adaptive system is portrayed 
in Figure 2: here the process is identified, the input signals are 
characterized, and the decision computer selects the controller 
characteristics which yield satisfactory performance during vari¬ 
ations of the input, the disturbances, or the process parameters. 
The problem is normally further complicated by constraints on 
the allowable values for u or constraints on the process state 
variables. 

The system of Figure 2 illustrates that a logical theory of 
adaptive control should be based upon the theories of: 

{a) Identification. 

( b ) Optimization. 

(c) Stability. 

Unfortunately, the theories in these three areas were notably 
inadequate six years ago when interest in adaptive control reach¬ 
ed an initial peak through the stimulation provided by the prob¬ 
lems of aircraft control over a broad flight regime and in an 
environment which was largely unpredictable and by the grow¬ 
ing interest in the analysis of adaptive biological systems with 
control and communication theories. Consequently, during the 
last few years major interest in control theory research has 
focused on the particular areas of identification, optimization, 
and stability—each an area of major theoretical and practical 
importance in its own right. 

The general optimal control problem was posed by Feld- 
baum in 1953 1 . Basically the problem is to determine a control 
input u which minimizes a given performance index ( S ) subject 
to certain constraints and given plant dynamics. The plant dy¬ 
namics, which relate the plant output v and state variables x to 
the control input u 9 are usually specified in the form 

x = f(x,u) (1) 

for continuous time systems or in the form 

X k+1 =f(x k ,u k ) (2) 

for discrete-time systems. 

In 1956 Bellman suggested the application of the theory of 
dynamic programming to the solution of optimal control prob¬ 
lems 2 . This theory leads to a first-order partial differential equa¬ 
tion for the solution of the optimal control input. The optimal 
control input is then obtained as a function of the current state 
of the system; hence we obtain a feedback structure for the 
realization of the optimum control system. 


At about the same time, a group of Russian scientists under 
the direction of Pontryagin was developing a theory of optimal 
control based on the classical calculus of variations—work which 
has become known as Pontryagin’s maximum principle 3 . A 
solution for the optimal control input in this case involves the 
maximization, with respect to u , of the function 

H= —F(x, u) + p T f {x , a) (3) 

where F represents the integrand of the performance index 
Ct 

S= I F(x, u)dt (4) 

J to 

and also involves the solution of a set of 2 n ordinary differential 
equations (where n represents the order of the state vector x). 
Boundary conditions are of the mixed type, specified both at t Q 
and T. 

A third approach to the optimal control problem was pio¬ 
neered by Kulikowski (1959) in the work on functional analy¬ 
sis 4 . In the United States, the functional analysis approach was 
developed by Kelley, Bryson, Ho and others under the general 
title of methods of successive approximations or method of 
gradients. In this approach, the control input u ( t ) is generally 
obtained as the limit of a sequence of control inputs, defined 
over the entire optimization interval (t 0 , T), which result in suc¬ 
cessively smaller values of the performance index S. The control 
input so obtained is generally of an open-loop nature (though 
a successive approximation technique which results in closed- 
loop control has been suggested by Bellman under the title of 
approximations in policy space). Thus, the computational ad¬ 
vantages usually associated with this approach must be weighted 
against the disadvantages associated with open-loop operation, 
although the advantages of closed-loop operation for the general 
non-linear system are largely intuitive. 

Recently, Kelley, Bryson and Breakwell have suggested 
gradient techniques to obtain nominal open-loop solutions com¬ 
bined with other techniques (maximum principle, dynamic pro¬ 
gramming) to obtain closed-loop operation from small devi¬ 
ations from nominal 5 . This approach is very attractive in the 
design of non-linear systems, since the feedback solution is ob¬ 
tained for a linearized system operating in the vicinity of the 
nominal solution. 

The theory of optimal linear systems 6 , which result when the 
plant is linear, the performance index is the integral of a quad¬ 
ratic form in x and u , and there are no constraints, is well de¬ 
veloped both for the stochastic and for the deterministic cases. 
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In this area of linear control, the concepts of controllability (can 
the plant be controlled with any input, optimal or not) and 
observability (can state x be inferred from measurements of the 
observable variables y) play special roles for multivariable sys¬ 
tems 7 . A recent paper by Kalman develops the beginnings of a 
relation between the optimal approach to linear systems and the 
classical design methods. It is shown, for example, that in order 
for a given linear control law to be optimal with respect to some 
quadratic performance index, the return difference must have a 
magnitude larger than unity over the entire frequency spectrum. 
The theory as developed thus far does not, however, permit quan¬ 
titative consideration of the usual practical problem in which 
the return difference magnitude cannot be made unity over more 
than a finite portion of the frequency spectrum. 

In addition to this general theory, research in optimum con¬ 
trol has considered a variety of special problems—e.g., the mini¬ 
mum time problem 8 (the evaluation of that control signal which 
effects a desired change in system state in the minimum time), 
or the vehicle control problem in which minimum final error is 
desired when the problem is complicated by uncertainties in 
observation of vehicle position and velocity and by limited total 
available quantities of fuel 9 . 

In the light of this general optimization theory, therefore, the 
adaptive control problem relates to the design of the optimum 
system in the presence of unpredictable variations in the state 
variables and with the controller designed to track, in some 
meaningful way, these variations. In the most general approach, 
the adaptive and optimizing aspects of the problem are not 
separated, but rather the problem is considered as a single entity: 
the optimum control of a stochastic or unknown process. In the 
linear case in which the state variables are corrupted by noise, 
for example, the solution is known to be an optimum estimation 
of the state variables from the observable variables, followed by 
the controller designed on the assumption that the state variables 
are known precisely. In the more general case, the control signal 
u must perform the dual function of effecting the desired control 
of y and simultaneously permitting continually improved esti¬ 
mation of the process parameters. 

The first extensive study of discrete-time stochastic control 
was published by Feldbaum in 1960; studies of continuous-time 
stochastic control problems appeared in the following year 10 . 
In all these cases, the complexity of the solutions realized seems, 
at least at the present time, to restrict applicability severely. It is 
generally assumed by the above authors that the stochastic vari¬ 
ables introduced in the control system result in a Markov pro¬ 
cess for the state variable x. When dynamic programming is 
applied to the continuous-time stochastic control problem, a 
second-order partial differential equation results. 

In an attempt to simplify the theory and bridge the gap be¬ 
tween general mathematical solutions and practical, engineering 
realizations, the optimization and identification problems have 
been considered separately. Identification is of importance in its 
own right as a fundamental element of control system design, 
since feedback control is primarily important because the pro¬ 
cess parameters are unknown precisely or vary with time or 
environmental parameters. Identification for adaptive control 
differs from the classical problem of network and control theories, 
however, because of the emphasis on minimizing the time re¬ 
quired to complete the determination of significant process 
characteristics. 


Most intensive studies of the identification problem have 
related to the evaluation of the impulse response of the process, 
or the matrix of impulse response functions (just as most practi¬ 
cal attempts to design adaptive controllers have operated from 
estimates of the changes in the various impulse response func¬ 
tions). Work has focused on the minimum time to determine the 
impulse response with smoothing of the measured data to pro¬ 
cedures for the determination of the maximum likelihood esti¬ 
mate—in both cases, with particular emphasis on the problem 
depicted in Figure 3 n . 


le5t _ 

inpuf 


Noise 


t r*v> 


Process to be 
measured 


Noise 


Figure 3. Identification problem 

Attempts to convert adaptive control theory to practical 
realizations have generally utilized a partial identification: e.g., 
determination of the damping ratio and/or the resonant fre¬ 
quency for a system known to be dominantly second-order, 
measurement of only the early portion of the impulse response 
corresponding to the period of time over which control is to be 
realized by the last control signal (with subsequent control sig¬ 
nals designed to override the effects of earlier signals and the 
later portions of the impulse response), or determination of only 
such gross features as the variation of the approximate duration 
of the impulse response or the settling time of the process. In 
such cases, which represent a segment of the bridge between the 
general theory and practical engineering realizations, very little 
theoretical analysis of accuracy, speed of identification, and 
other aspects seems to have been reported in the control literature. 

Just as the identification problem can be severed from the 
theory of adaptive control for analytical and design purposes, 
the stability considerations can be separated from the design of 
the adaptive loop. Until the interest in adaptive control arose 
six years ago, the control engineer was predominantly concerned 
with the classical stability analyses of linear, time-invariant sys¬ 
tems according to the familiar graphical and analytical criteria 
of Routh, Hurwitz, Nyquist, etal., or with the analysis of non¬ 
linear systems via the phase portrait, the analysis with describing 
functions or other approaches to quasi-linearization, or direct 
linearization procedures. 

At approximately the same time as interest arose in adaptive 
control, Liapunov stability theory came to the attention of the 
control engineer, and serious efforts were made both to analyse 
and to design non-linear control systems of practical engineering 
importance 12 . While a variety of the optimization procedures 
which have been developed lead inexorably to stable systems 
through the choice of a performance measure which results in 
satisfaction of the Liapunov conditions, the adaptive systems 
which have been built often are described by such complex non¬ 
linear differential equations as to discourage any attempts at a 
theoretical study of stability. Extensive analogue and/or digital 
simulation and testing provide the only available, albeit mild, 
assurances of system stability. The difficulties arise, of course, 
because of the time delays associated with identification (whether 
or not the identification portion of the system be separated from 
the controller), the multiple non-linearities represented by the 
constraints and by the adaptivity of the controller, and the time- 
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varying nature of the system—all of these characteristics combin¬ 
ed with the high order of most realistic systems of engineering 
interest. While studies of simplified approximations to specific, 
actual systems have indicated in several cases the possibility of 
instability, the prospect is certainly not hopeful for meaningful 
studies of actual systems; it appears that stability theory, as it 
develops during the next three years, will primarily be useful for 
the increased understanding of the dynamic properties of rela¬ 
tively simple adaptive systems. 

Thus, the theory of adaptive control is evolving, built largely 
upon the three distinct areas of optimization, identification, and 
stability. A fourth focal aspect, that of sensitivity, although of 
apparently equal significance, has been developed only super¬ 
ficially. Several facets of the sensitivity problem are illustrated 
by the simple adaptive system of Figure 4. Here the controller is 
varied automatically to minimize a performance measure in the 
presence of slow variations of the process parameter g. (The g 
may be a particular physical parameter or a derived character¬ 
istic, such as the deviation of the process from a reference model 
or a relative damping ratio.) 



Figure 4. Simple adaptive system 


Two quite different aspects of the sensitivity are of interest 13 . 
First, we may consider the sensitivity of the overall system trans¬ 
mission (or the performance measure) to the changes in g; in 
such a case, the adaptivity can be interpreted as a technique for 
reduction of this sensitivity, and design and evaluation of the 
adaptive loop can be effected from this sensitivity viewpoint. 
Alternatively, we may be interested in the sensitivity of system 
performance to changes in other process or environmental para¬ 
meters—as a measure of the overall merit of the adaptive system. 
In both cases, the close relations between sensitivity and stability 
for linear systems (the relations which indicate the parameter 
margins for the system stability, or how much specific para¬ 
meters can vary before the system breaks into oscillation) pro¬ 
vide a basis for stability studies in the presence of parameter 
variations. 


Adaptive Controllers 

Within the realm of practical adaptive control, we include 
those design approaches which focus on the concept of improved 
system performance, and which attempt to realize this goal by 
restricting consideration to a sub-optimal problem of system de¬ 
sign. For example, within this category fall those systems which 
operate on the basis of the adjustment of a single parameter 
(or a small number of parameters) to extremalize a selected func¬ 
tion of system performance. The system configuration and the 
individual elements are selected according to conventional con¬ 
trol techniques, after which the optimization is included in order 
to improve performance within the framework of the original 
system design. 

The technical literature of feedback control is replete with 
examples of such sub-optimal systems developed during the last 
few years. In general, such designs are characterized by simplicity 
both conceptually and practically, reliability, and emphasis on 
practicality. In terms of control systems theory, such designs 
represent major contributions primarily in terms of the novel 
configurations which results—systems which never would evolve 
from the conventional control theory focusing so heavily on 
stability considerations for linear, single-loop feedback configu¬ 
rations. 

The research efforts along this direction of specific adaptive 
systems can be illustrated by three specific approaches, the first 
of which is depicted in Figure 5, a sketch of the M.I.T. Instru¬ 
mentation Laboratory adaptive system which has evolved from 
the early work by Draper and Li on optimalization. In this 
configuration, a conventional feedback control system (represent¬ 
ed in simplified form by the single control loop in the figure, 
even though in most cases the configuration is multi-loop) is 
designed; superimposed on this is the optimizing sub-system. 
The actual system output is compared with the response of a 
model to yield a generalized error e g . A non-linear function of 
this e g is used to adjust a parameter K of the control loop to 
yield a minimum of the performance measure. 

Certain basic problems arise in the design of such a model- 
reference adaptive system: 

(1) How is the model to be chosen? 

(2) How are we to select that parameter K which is to be varied ? 

(3) How is the performance measure or criterion to be chosen? 

(4) Under what conditions is the adaptive loop itself stable? 

While certain aspects of these problems have been investigated 14 , 
the extension of this approach to complex configurations relies 



Figure 5. Model-reference adaptive system 
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heavily on. the final design and verification of performance 
characteristics via simulator studies. Such a situation should be 
neither surprising nor discouraging, however, in view of the fact 
that optimization techniques are particularly useful in the design 
of complex systems in which the engineer is faced with the diffi¬ 
cult question of how to start. 

The extensive development of the configuration of Figure 5 
has been motivated, at least in part, by the adaptive autopilot 
problem for piloted aircraft moving through radically varying 
environments—a specific engineering problem which also stim¬ 
ulated the development of the Minneapolis-Honeywell auto¬ 
pilot configuration 15 . In this system, the controller gain is ad¬ 
justed automatically to maintain a small-amplitude limit cycle 
of a suitably chosen process variable. In more recent work, a 
digital computer is utilized as the controller to provide digital 
compensation, to furnish the required forward loop control, and 
to simulate the automatic gain changer, the model, and the gyro 
blender and circuitry which is used to control the body-bending 
modes. This particular system typifies the strong trend today 
toward digital realizations of adaptive controllers. 

Complementary to the aircraft industry’s interest in adap¬ 
tivity has been the emphasis on industrial applications of com¬ 
puter control: once the computer is installed for data logging 
and routine processing of system performance information, for 
shutdown, start-up and emergency control, and for the auto¬ 
mation of routine economic and profit analyses, the systems 
engineer visualizes utilization of the computer flexibility to real¬ 
ize optimum plant performance, or at least control which is 
more precise and faster than that achievable with human oper¬ 
ators. 

In the technical literature describing applications of optimum 
computer control, two approaches dominate: the model ap¬ 
proach and the automatic experimental approach 16 . Under the 
former design philosophy, a model of the process is used to de¬ 
termine the optimum control signal as a function of measurable 
signals and disturbances. The response can be very rapid, can 
avoid difficulties with multiple extrema, and can include learning 
or updating modification in the model; on the other hand, per¬ 
formance is limited by the accuracy of the model, and realization 
of performance near optimum requires intensive studies of and 
measurements on the process to be controlled. 

In the automatic experimental approach, the optimum is 
realized by the injection of artificial input signals to perturb the 
operating point, with a subsequent evaluation of whether the 
performance improved or deteriorated as a result of the change. 

In such an approach, the speed of response is limited by the fact 
that the' system must search for an optimum, the presence of 
multiple extrema causes difficulties, and the existence of many 
signal variables leads to exceedingly slow and possibly poor 
performance. A considerable portion of the optimization litera¬ 
ture of the past few years is devoted to study of searching tech¬ 
niques to overcome one or more of these difficulties. 

Chen and Decker emphasize the advantages to be gained 
by combination of these two approaches, with the performance 
of the composite system indicated in Figure 6. The plot shows 
the payoff J as a function of the control signals u for various 
disturbance inputs ( d ± and d 2 here). The solid curve portrays the 
variation of J with u for the actual plant, the broken curve the 
corresponding solution for the simplified model of the plant. 
The two constraints indicate the allowable bounds on u and / 


which result from considerations such as safety or which are 
imposed to avoid subsidiary extrema. 

If the system is initially operating at point 0 with a disturbance 
input d l9 and the dynamic performance is demonstrated by a 
change from d x to d 2 , we find that the performance moves initial¬ 
ly to point E since u cannot change instantaneously. As fast as 
the model system responds, operation moves to point C; there¬ 
after the automatic experimental procedure moves the system 
toward the optimum operating point B. Clearly we are using the 
model here for fast, gross corrections, the automatic experimen¬ 
tal system for the slower, fine corrections, although the specific 
interrelation of these two portions of the system may be quite 
complicated in problems more realistic than the simple situation 
depicted in Figure 6. 



u 


Figure 6. Example of restricted optimization and adaptivity 

The final aspect of restricted and simple adaptive control to 
be discussed is based upon the utilization of simple digital logic 
for the determination of the desired control signal u on the basis 
of inspection of the response of a high-speed model of the pro¬ 
cess. If for conceptual simplicity we assume the discrete case, we 
can consider the problem of selecting the control signal which is 
to be constant over each interval of time. At the time * = 0, we 
wish to determine the u for the first interval from 0 to T. This 
determination must be derived from the available information: 
the present and past values of the control signal, the response x, 
and the reference input r. 

The determination can be implemented in the following way. 
We consider only the values of r , x, and u every T seconds (i.e., 
at the sampling times). On the basis of the known statistical 
characteristics of r, we can estimate r l9 r 2 ,... On the basis of the 
past values of u and x and our knowledge of the process dy¬ 
namics, we can determine the response values into the future 
(xi, x 2 , x 3 , ...) for any assumed sequence of control signal values 
*•** If we wish to minimize the mean square error, we 
might attempt to select u 0 in such a way that (assuming u l9 u 2i ... 
are later selected optimally) we would minimize the summation 

(r 1 -x 1 ) 2 + (r 2 -x 2 ) 2 + ... 

It seems apparent that the choice of u 0 has a decreasing effect 
on the successive terms in this series; furthermore, because of 
the increasing difficulty of predicting r$ as we look farther into 
the future, we should weight more heavily the early terms. Such 
considerations suggest the consideration of only a small number 
of these terms, for example three: 

J = (r 1 -x 1 ) 2 +(r 2 -x 2 ) 2 +(r 3 -x 3 ) 2 (5) 

Here r$ are predicted values of r; the Xj are future system response 
values which incorporate the effects of past signals plus the 
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effects of the variable or controllable future signals u l9 and u 2 . 
The optimization involves determination of the minimum of J 
over the allowable range of values of Uj, but we actually apply 
to the process only the signal w 0 . When t = T (and % is to be 
applied), the new optimum value for the control signal is to be 
re-evaluated. 

The attractiveness of this approach to sub-optimization de¬ 
rives from the possibility of simplification in the evaluation of 
u 0 . If we consider a binary control signal (with u either — 1 or 
+ 1), we must choose between these two values on the basis of 
minimization of /, where we assume u x and u 2 will subsequently 
be selected in an optimum manner. In terms of the logic required 
to implement this decision on w 0 , we need to divide our three- 
dimensional space into two parts: one requiring a u 0 of— 1, the 
other + 1. Our present location in this three-dimensional space 
is determined from the three predicted values of the future sys¬ 
tem input and the response with all possible control signals. 

Major further simplification is possible if the system is linear, 
so that the future response can be divided into two, additive 
parts resulting from past inputs and from future inputs 17 . Then 
our present location in this three-dimensional space is determin¬ 
ed by the predicted values of the future system error with no 
control input. Implementation of this control scheme involves 
only high-speed prediction of e l9 e 2 , and e% without control, and 
then (by simple digital logic) determination of the location of 
this state with respect to the division of state space into the two 
parts corresponding to the two possible control signals. Thus, 
this system involves only a high-speed model of the process to 
predict future response values from the present energy storage 
conditions, a predictor to act on the present and past input, and 
simple digital logic to determine in which half of the three- 
dimensional space we are situated. 

The three schemes described in this section are only three of 
a wide variety of practical system realizations derived with 
emphasis on a sub-optimization approach to adaptive control. 
In each case, design of the optimizing equipment requires at 
least a reasonable estimate of process dynamics; in each case, 
the optimizing system is designed to correct for slow variations 
or the effects of low-frequency disturbance inputs; in each case, 
the optimizing components are sufficiently simple to permit 
simultaneous realization of control over reliability. These ad¬ 
vantages are at least to some extent offset by the fact that mean¬ 
ingful analysis of the three systems is apparently not possible if 
we are working with anything other than the most elementary 
processes; actual design and verification of the value of the 
optimization and adaptivity must rest upon computer simulation 
studies and actual equipment tests. 

Contributions of Adaptive Control Research 

In these final paragraphs, it is appropriate that we summarize 
briefly at least some of the contributions of the intensive work 
in adaptive control theory during the three years since the First 
I.F.A.C. Congress. Obviously, it is not possible in such a dis¬ 
cussion to separate the work in adaptive control from that in 
stability theory, optimization, identification, sensitivity, and so 
forth, since these various aspects of control theory are so totally 
interdependent. 

First, what are the fundamental contributions to control 
theory? Optimization theory provides a fundamental scientific 


basis for control system design. The optimum design serves as a 
yardstick for quantitative evaluations and interpretations of all 
of the conventional control system design techniques. In a 
wide variety of linear problems, optimization theory provides a 
straightforward approach to system design. In an entire class of 
problems, for example, the theory indicates that the optimum 
controller is a saturating amplifier and specifies both gain and 
saturation level. More generally, the easily determined saturat¬ 
ing amplifier may require an associated, linear, dynamic control¬ 
ler; even when these linear transfer characteristics are easily de¬ 
termined only very approximately, the theory indicates the very 
important structure of the optimum solution. In other problems, 
the selection of a suitable Liapunov function as the integrand 
of the performance measure leads directly to a design criterion 
and solution, with the latter inevitably stable. 

Thus, optimization research and the consequent adaptive 
research have provided an entirely new group of theoretical 
solutions. In addition, and perhaps of equal importance, the re¬ 
search has focused attention on an entirely new set of concepts. 
Three years ago we were only vaguely characterizing systems in 
terms of controllability and observability, a vagueness which 
carried over from intuitive understanding based upon earlier 
network and feedback theories. Interest in adaptive control is 
leading to extensions of sensitivity theory and has generated an 
interest in identification which transcends the narrow and highly 
restricted concern earlier of the communication engineer and 
the control engineer. 

Perhaps the most important contribution of adaptive control 
has been the wealth of entirely new systems which have evolved 
from the research—systems based on digital logic or the entire 
group of model-reference configurations: control systems which 
can never be derived from the ‘classical’ control theory of 1950. 
The occasional arguments we still hear today from the ‘clas¬ 
sicists’ who claim that adaptive control can achieve nothing un¬ 
obtainable with conventional feedback theory lose sight of the 
essential nature of engineering: the continual search for novel 
design approaches. The availability of three solutions to any 
engineering design problem is always better than having only 
two; adaptive control has opened up a host of new, unconven¬ 
tional, and exciting solutions. 

Any overall evaluation of the significance of adaptive control 
research must recognize the impact of this research on engineer¬ 
ing education and the resulting changes, both today and in the 
future, in the capabilities and breadth of interest of engineers. 
In particular, control research and engineering emphases have 
been primary factors in the early introduction into educational 
piogrammes of the concepts of state models and numerical 
methods. In the former category we should include the relations 
among state models, transfer functions, flow-graph representa¬ 
tions, and analogue and digital simulations—relations which 
unify so much of system analysis, including (as an example) the 
various approaches to linear-system stability. In the latter cate¬ 
gory, we must encompass digital simulation and computer 
control, as well as numerical techniques for the solution of 
typically complex analysis, evaluation, and identification pro¬ 
blems. 

Finally, the interest in adaptive control reflects most poign¬ 
antly the interrelationships between control engineering and 
such areas as economic system analysis, pattern recognition, 
learning theory, medical engineering, and bioelectronics. The 
term ‘adaptive control’ was first borrowed from the biological 
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field; the control engineer’s interest in the engineering and scien¬ 
tific theories of adaptive control is paralleled by the biologist’s 
and physiologist’s growing interest in the control field. 

The author is deeply indebted to the other members of the con¬ 
trol research group at the Polytechnic Institute of Brooklyn , and 
particularly to Professors R. Drenick, P. Dorato , andL. Braun, Jr. 
The support of the U.S. Air Force Office of Scientific Research 
under Grant AF-AFOSR-62-280 is gratefully acknowledged. 
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Summary 

Learning machines have increasing practical importance and it is 
necessary to develop a competent theory for these systems rather than 
treating them as though they were simply an intractable kind of adap¬ 
tive controller. Since learning is a goal directed change in a pattern 
of behaviour, it is a property that depends upon the environment and 
the objective of the machine, as well as the specification of the machine 
itself. Briefly, it is a property of an organization and, consequently, 
cybernetics offers the most suitable framework for a theory of learning 
machines. 

Present-day learning machines, which may be large and elaborate 
networks of adaptive components or the equivalent computer pro¬ 
grammes, stem from the pioneer work upon conditional probability 
machines and homeostatic systems. The mechanical concomitant of 
learning always involves structural modification, over and above adap¬ 
tive change in the system parameters, but the modifying process, al¬ 
though it may involve ‘random’ variation, is well specified in any 
realistic device (purely ‘random’ search is impracticable). Further, a 
useful learning machine has an hierarchical organization. If it is part 
of a control system there will be different levels of goal associated with 
different levels of control. In any case there will be more or less abstract 
representations of the environment. When the machine is required to 
learn about a symbolic or linguistic environment (in problem solving 
or in man-machine symbiosis) these levels correspond to metalanguages 
in terms of which internal data processing is carried out and external 
communication can readily take place. An evolutionary learning 
machine is able to build its own hierarchical structure and in some 
conditions this process may be identified with a form of ‘concept 
acquisition. In order to describe the underlying mechanism, it is neces¬ 
sary to introduce the idea of ‘reproducing’ or unlocalized automata. 

Where design principles are concerned, brains and neural structures 
still provide the most useful data, though there is no necessary relation 
between the structural or functional specification of a brain and of a 
learning machine. Certainly there are features of development, like the 
need for maturation in contact with the environment that are common 
to each system. On the assumption that the biological analogy contains 
a fair grain of truth, the realization of tangible learning machines will 
call for large flexible and computationally ‘parallel’ assemblies of 
coupled transmission lines. It has been proposed that these active 
transmission lines would be optimally embodied in niacromolecular 
structures and this suggestion appears to be physically plausible. 


1. Introductory Comments 

1.1. Importance of Learning Machines 

Learning machines are more than the ingenious toys they are 
commonly supposed to be. They constitute a large and important 


class of automata. Members of this class that have been realized 
as artefacts (mechanical or electronic models) provide existence 
proofs, often the only existence proofs available at the moment, 
for propositions in the theory of learning machines. The theory 
of automatic control and finite automata (devices such as switch¬ 
ing networks that compute a function of a well defined input) is 
a beautifully constructed but highly specialized fragment of the 
theory of learning machines. 

Beyond a certain limit, the fragment of a theory cannot be 
usefully extrapolated. Conceptually plausible analogies with 
familiar constructions break down and it becomes more con¬ 
venient to tackle the problems that arise in terms of a framework 
that embodies the original and well-known area of study as a 
special part. My contention is that this critical stage has been 
reached in connection with many problems of automatic control. 
Attempts to describe essentially learning situations in terms of the 
conventional formulations are often strained, or inadequate, or 
beset with illegitimate approximations. Whether he likes it or 
not, the control engineer is being forced to accept the need for 
a theory of learning machines and not infrequently he is also 
required to develop a theory where none exists. 

In this case, learning machines, however crude they may be, 
provide valuable indicators (some of them suggest solutions to 
particular problems; for example, in pattern recognition), but 
the mere existence of learning machines is insufficient. The pre¬ 
sent theoretical background (which is firm but far from elegant) 
needs rapid development in order to serve a new cybernetic 
technology of control and computation in systems previously 
regarded as intractable. 

To restrict the discussion, obviously trivial learning machines 
(tape recorders with cunningly manipulated inputs and outputs) 
will be excluded even though they are presented in a plausible 
guise. Next, non-trivial learning machines will be excluded if 
they are more opportunely described in some other fashion 
(adaptive systems like Gabor’s 1 genuinely Team’ about a re¬ 
stricted environment but their activity can be analysed within 
the conventional theory of control and ‘learning’ need not be 
considered). One can omit the field of finite automata, in which 
the work of McCulloch and Pitts 2 on neural network analogues 
is pertinent to brains, but oriented towards computation rather 
than learning. Finally, for lack of space rather than relevance, 
a detailed examination of the Teaming algorithms’ that feature 
in ‘artificial intelligence’ is avoided although it will be neces¬ 
sary to view the field rather broadly in 2.8. There are still, 
to my knowledge alone, over 350 significantly different learning 
machines which have either been simulated by computer pro¬ 
grammes or constructed as special mechanisms. 

1.2. Deterministic and Probabilistic Systems 

A system, whether it is an animal or a machine, is always 
specified in relation to a particular method of observation. In 
the case of animals this specification usually amounts to the ex¬ 
perimental conditions, the environment in which the animal is 
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placed (a rat in a maze) and the measurements it is proposed to 
make (whether the rat turns left or right). In the case of a ma¬ 
chine the specification is often dictated by the circuitry and, in 
so far as control mechanisms are concerned, the environment to 
be controlled. At any rate, whenever a system is mentioned there 
is also, by definition, a state description that lists the relevant 
states, which, in the ultimate analysis, are states which, for often 
excellent reasons, we, the experimenters, deem relevant. 

A behaviour is a sequence of output states (or actions) that 
are contingent upon input states or evidence and upon the inter¬ 
nal state of the system. If a behaviour is repeatable and consistent 
it can be ascribed to a particular computation that the system 
performs (in other words, in view of the observed regularity, the 
system can be described as though it had been built to compute 
a given function of its input). Of course, if the system is a 
machine that has been built for some purpose, it will be possible 
to infer this from an inspection of its circuit. Statistical invariants 
are countenanced as regularities and if these exist, even in the 
absence of any deterministic pattern of behaviour, the system is 
said to compute a probabilistic function of its input. 

There is partial ignorance of the internal state and the 
exhaustive input specification of men and animals, their experi¬ 
mental gambits are only defined at the probabilistic level. For 
constructed machines, with the exception of some evolutionary 
systems introduced in 2.7, a deterministic specification is pos¬ 
sible. It may, however, be completely impracticable, because the 
system is very large (networks in 2.4), because there is a 
‘redundancy of computation’ so that the same easily recognized 
behaviour may be due to the action of different mechanisms (as 
in many stable ‘error correcting’ computers), or because internal 
parameters of the system are changed as a function of its previous 
states in such a way that it proves impossible to assume a mo¬ 
mentarily static structure. 

Adaptive machines with a ‘black box’ for providing an un¬ 
correlated forcing input that modifies their parameter values, for 
example, the adaptive controller in Figures 6 or 7 and some of 
the machines described in 2.3 and 2.4 are strictly deterministic 
if the interior of the ‘black box’ is examined but it is useful to 
describe them as though they were probabilistic devices if the 
sequence of values of the forcing input is irrelevant (as it is when 
the machine is intended to control a statistically defined environ¬ 
ment). Nor are ‘random’ learning networks (like those described 
in 2.4) really probabilistic, because their initial connectivity is 
determined by a particular ‘random’ number table. In this case, 
however, a very different probabilistic fiction is sometimes use¬ 
ful. The ‘random’ number table is chosen to have certain statis¬ 
tical constraints and the particular network is regarded as a 
‘random’ sample from a statistical ensemble characterized by 
these constraints. Whereas the sequence of input events was 
previously deemed irrelevant to the adaptation of the given 
machine, in this case the variations of the machine within the 
statistical ensemble are deemed irrelevant. 

1.3. Minimal Criterion for Learning 

A minimal criterion for learning is a goal directed change in 
computation or possibly ‘a novel computation’. By analogy, 
when one says that a man ‘Z’ has learned a skill or learned a 
telephone number one does not merely mean that some change 
has occurred in his brain (it certainly has, of course, and some 
kind of adaptation is a prerequisite for learning). One also means 


that the man has acquired the computing process involved in 
performing the skill or reciting the telephone number. 

Further, since learning is a property of a system’s behaviour, 
an immediate inference from 2.2 is that it will depend upon 
the experimental conditions (this machine will be a learn¬ 
ing machine in some circumstances but not in others). The man 
‘JP, for example, can only be said to have learned if he is present¬ 
ed with an environment in which he can recall his skill or his 
telephone number. 

Finally the property of learning is unaltered in an isomorphic 
model or representation of the system since it does not directly 
depend upon material construction. Nobody argues that men 
learn because they are made of protein or that computers cannot 
learn because they are made from metal. Rather, ‘learning’ be¬ 
longs to an organization some part of which may be common to 
men and computing machines. To summarize these points one 
can say that ‘learning’ is a cybernetic construct and is capable 
of embodiment in cybernetic models that represent organizations. 
The condition that learning is goal directed implies that an ob¬ 
server or model builder can understand the change in computa¬ 
tion that takes place as a result of learning and restricts the dis¬ 
cussion to non-trivial cybernetic models. 

In fact, the most interesting kinds of learning involve more 
specialized changes than have been described. A system may 
learn to abstract features of its environment (in other words, to 
represent a class of events in terms of attributes of this class). 
It may associate equivalence classes of its internal states (called 
‘signs’) with subsets of points in a descriptive space which has 
these features or attributes as its coordinates, when the ‘sign’ 
may be called a symbol and the subset of descriptive points, its 
denotation. The system may combine symbols or form classes 
of invariant relations between symbols (which, in the logician’s 
sense, can serve as universal). Finally, one might (in some care¬ 
fully selected circumstances) refer to the process whereby inter¬ 
nal or external events are signified and manipulated as ‘percepts’ 
and ‘concepts’. 

A great deal of machine learning does involve processes that 
appear to be isomorphic with ‘concept acquisition’ (using this 
phrase like those objectively oriented psychologists who aim for 
a functional account of mentation). Whether a given observer is 
prepared to say that the machine concerned does acquire con¬ 
cepts, or, indeed, whether he is prepared to say it learns (rather 
than ‘learns’) is a matter, as MacKay 3 points out, for that ob¬ 
server to decide. The decision has very little to do with the 
manifest behaviour and chiefly depends upon his personal view 
about how much he resembles the machine 4 . All a cybernetician 
can assert is that for some systems there is no mechanical ab¬ 
surdity involved in the contention that the device is able to 
learn or conceptualize. So far as the minimal criterion is con¬ 
cerned, at any rate, there is a tacit assumption that a ‘learning’ 
machine is literally accepted as a learning machine. 

1.4. Machines that Imitate Organisms 

There are two broad categories of learning machine: 

(1) Devices that imitate the undeniable learning behaviour of 
an organism. 

(2) Devices that manifest the abstract cybernetic property of 
learning. Frequently, the apparatus must also satisfy a practical 
goal like learning to recognize distorted alphabetical characters 
or learning to control a non-stationary process. 
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In either case, a simulation (a tangibly realized cybernetic 
model) must embody salient features of the environment as well 
as the learning system itself. 

Members of category (2) are fairly well known and are con¬ 
sidered at length in 2.2. For the moment let us examine a few 
members of category (1), which are less familiar in the discipline 
of engineering. 

The learning machines of (1) simulate animals or brains or 
parts of brains. Typical small animal simulators are Deutsche’s 
rat 5 (which ‘learns’ to run a maze according to the tenets of 
Deutsche’s learning theory) and Angyan’s turtle 6 (which ‘learns’ 
about a more liberally specified environment, using the strategies 
of sequential conditioning). In each case, the behaviour of a real 
animal is abstracted and the inbuilt mechanism constitutes a 
psychological or neurophysiological hypothesis about the gener¬ 
ation of learning behaviour which is tested and explicated by 
embodiment in an artefact. From the viewpoint of control 
engineering, their chief value stems from the argument given 
in 3.1, namely that although there is no necessary connection 
between learning machines and brains or animals, the most 
potent clues about the design of learning machines come from 
biology. 

The first of these models was constructed by Grey Walter 7 
in the form of a mechanical ‘tortoise’ with auditory, visual and 
tactile sensory inputs and motor capabilities that drove it around 
the floor with a behaviour initially determined by inbuilt light 
responsive reflexes. Grey Walter used this machine to demon¬ 
strate the acquisition of conditioned reflexes. In particular, he 
aimed to show the consequences of a four stage learning hypo¬ 
thesis. 

(a) Signals representing unconditioned stimuli (in other 
words, stimuli that already give rise to some response, due to 
built-in association) are abrupt and demark the onset of a stimu¬ 
lus, whereas signals representing novel stimuli (which may be¬ 
come conditioned stimuli) are ‘stretched’. 

(b) The overlap between unconditioned and (already ‘stretch¬ 
ed’) novel stimuli is counted as a measure of the regularity of 
this joint event. 

(c) If the overlap count exceeds some threshold value an 
autonomous process, such as a damped long term oscillation, 
is initiated. Very tentatively, this event corresponds to an ‘in¬ 
sight’. 

(d) If the novel stimulus is combined with the existence of the 
autonomous oscillatory process that indicates frequent coin¬ 
cidence between the unconditioned and the novel stimuli, the 
novel stimulus becomes coupled to and will elicit the existing un¬ 
conditioned response. Hence, the novel stimulus becomes a con¬ 
ditioned stimulus. 

Learning machines that simulate the activity of a brain differ 
according to the degree of physiological verisimilitude and the 
level of abstraction involved in their design. At one extreme 
there are models, like Harmon’s artificial neurones 8 , that ac¬ 
curately replicate the characteristics of a real neurone (in so far 
as these are known from microelectrode recordings). Small col¬ 
lections of these elements (corresponding, at most, to ganglionic 
organizations) exhibit versatile behaviour patterns, some of 
which have properties akin to learning. At the other extreme 
there are statistical models, such as that of Beurle 9 ’ 10 , in which 
a large collection of artificial neurones, characterizing a more 
abstract image of the real neurone than those designed by Har¬ 
mon, are connected to form a network that replicates the statisti¬ 


cally defined connectivity of the mammalian cerebral cortex. If 
one of Beurle’s self-oscillatory models is coupled to a suitable 
environment it, also, manifests learning behaviour. 

At this level, the systems of (1) and (2) above are distin¬ 
guished by intention alone and most of them are open to a dual 
interpretation. Thus one of the conditional probability machines 
of Uttley 11_15 , which are described in detail in 2.3, can be 
arranged to imitate a brain very closely, but its value does not 
depend upon imitation. A conditional probability machine is 
interesting in its own right. 

1.5. The Material and Symbolic Dependence of a Realizable and 
Effective Learning Machine 

Although the fabric of a learning machine, synthesized on 
the basis of a cybernetic model, is irrelevant to learning (so that 
there is no need to distinguish between a computer programme 
and a special purpose device) it is not true that naturally occur¬ 
ring machines (animals and other organisms) are independent of 
their material character. The natural and the synthetic machines 
are both said to learn because their computation changes, but 
an animal changes its behaviour in order to survive in a change¬ 
ful environment. Hence it learns because it, or its species, ‘must 
learn’. Its ‘goal’ entails its existence. Further, its ‘goal’ is fashion¬ 
ed by the material used in its construction. In a constructed arte¬ 
fact the goal is introduced as part of the cybernetic model. 

Now if the learning machine is supposed to imitate an ani¬ 
mal, the goal will be an abstraction of certain energetic or ma¬ 
terial constraints that act upon the real animal. Ashby 16 represents 
these constraints as a set of ‘essential variables’ (imaging, for 
example, the temperature and blood sugar concentration in a 
real creature) whose values must, in a cybernetic model, be main¬ 
tained within prescribed limits. On the other hand, if the model 
simulates the property of learning (without any biological over¬ 
tones) the goal is somewhat arbitrary and in practically applicable 
models it is often determined by a desire to optimize process 
parameters or to recognize specific figures. The important point 
is that we do not have the liberty in choice of a goal that a cyber¬ 
netic model suggests. No realisable machine can neglect the con¬ 
straints and continuities of a material system. 

The point is self-evident so far as simple adaptive controllers 
of the kind considered in 2.2 are concerned. No sane man de¬ 
signs a completely hapless servomechanism, but in this simple 
case, the machines inhabit a well-defined domain. Their goal, 
amongst other things, is always obvious. The physics of the situ¬ 
ation imposes a partitioning upon the environment that deter¬ 
mines a reasonable path whereby the machine can achieve one 
after another in an hierarchy of subgoals and ultimately satisfy 
the main objective. To avoid any confusion, let us reaffirm that 
the information conveyed by a message, coded as a signal 
sequence, is substantially independent of energy in a macroscopic 
system. The wise designer, however, gives up the freedom he 
might exercise and specifies an interaction between the machine 
and its environment wherein the form of message adheres to 
the form of energetic or material constraints. 

Many of the most useful applications of learning machines 
(in pattern recognition, the control of really large industrial pro¬ 
cesses, in management, mechanical translation, man machine 
symbiosis and in library retrieval), refer to a symbolic or 
linguistic universe of discourse which constitutes an almost un¬ 
charted environment wherein the designer has far greater free¬ 
dom than either nature or a prudent engineer. Even in the most 
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favourable cases where the goal is well defined (as it is in pattern 
recognition and is not in management) there is no guarantee 
that algorithms exist for achieving this goal or that partitions 
exist in the set of possibilities. 

These issues are, of course, interdependent and Aizerman 17 
in his paper, cites a case where a character recognition machine 
can be designed, with reasonable goal approaching algorithms, 
because the symbolic environment satisfies a ‘compactness hypo¬ 
thesis’ which implies the possibility of partitioning (Aizerman’s 
‘compactness hypothesis’ reflects a cultural constraint analogous 
to the energetic constraints of the servomechanism designer. 
Other principles encountered in this field reflect linguistic, psy¬ 
chological and logical constraints). 

It can be shown that learning machines are able to perform 
abstractions and manipulate some kind of ‘concept’ 18 . It is argued 
in 2.4 and 2.5 that they can build non-trivial descriptive hier¬ 
archies, associated with ‘conceptual’ levels and directly related 
to the subgoals that are searched for in terms of a given level of 
abstraction. All this is a prerequisite for successful communica¬ 
tion with a symbolic environment but it is not enough to ensure 
effective control. In a given symbolic environment only some 
concepts’ can be generalized and, since generalization is the root 
of coherent inference, only some rather specialized concepts are 
useful (in the sense that only these yield mechanized inference in 
place of a haphazard and utterly impracticable search for the 
solution to a problem). A simple but adequate device is con¬ 
sidered by Ullman 123 . 

As a conjecture, any knowable symbolic environment is 
regular enough to admit inference by a machine that embodies 
the principles that determine its regularity (for, as Greene 19 so 
often points out, a learning machine cannot be a passive entity 
that is shaped by its experience. It must, if it learns by inference, 
have embedded preconceptions about its environment). It may 
turn out that the embodiment of these principles of regularity, 
like the evolutionary rules given in 2.7 und 2.8, depends upon 
a basic property of physical networks. At least one can cite man 
as the natural existence proof of such a machine. 

2. Machine Organization 

2.1. The Organization and the Environment of Learning Machines 

Apart from citing a few illustrative cases the first part of the 
discussion is concerned with information and computation, with 
the organizations that compute, their input and their output. 
Later one will examine the physical processes that mediate this 
organization. Since the minimal criterion for learning is stipu¬ 
lated in 1.3 as a change in the computation that characterizes 
a behaviour pattern, any learning machine ‘learns to compute’. 
Since this change must be goal directed (again from the criterion 
given in 1.3) any learning machine also ‘learns to control’ 
(even if only in the rather trivial sense that it controls the rein¬ 
forcement delivered by the experimenter who acts as its environ¬ 
ment). 

Different kinds of learning machine will be distinguished ac¬ 
cording to the control procedure they are able to learn, the 
form of environment in which they learn, and the goals they 
achieve. 

2.2. Well Defined and Nearly Stationary Kinds of Environment 

Suppose that the environment has the characteristics of a 
closed (or approximately closed) physical system such as an 


industrial process. It is described in terms of variables like tem¬ 
perature and pressure that are well defined, often continuous and 
always related by process equations. The goal may be to maximi¬ 
ze the output of a product. If the system has a stationary be¬ 
haviour (when it really is closed and its equations really are 
complete) the learning machine is employed for convenience 
alone. Its designer could have learned the equations himself and 
built them into the control mechanism. On the other hand, if the 
system has a non-stationary behaviour, the controller must adapt 
its form of computation to maintain the required goal condition. 
It must be an adaptive controller and may, perhaps, be a learn¬ 
ing machine. Andrew 20 has discussed the subject in detail. 
Although the structure of an adaptive controller is well known, 
it is cited in Figure 1 for completeness and in order to introduce 
a symbolism. 



Figure 1 


The overall controller B inspects the input and output state 
(the outcome or environment state) of a computer C that con¬ 
trols the environment. B determines proximity to a given goal 
as a value of 6 (x, y), a descriptor, and changes the function 
computed by C (as indicated by the parametric coupling) so that 
the goal state is approximated. The learning machine B, C, is a 
deterministic or stochastic hill climber in which C obeys a rela¬ 
tion of the form 

(l) 

where x e X is an input state, y e Y is an output state, E is a 
function fe F with an index </> adjusted by the output of B and 
where y designates a selection y t+1 if x is a selection x t9 t— 1,2,... 
The box B obeys a relation 

0 = G(x,y) (2) 

where C? is a strategy chosen to maximize the value of 0 (x, y). 
Minsky and Selfridge 21 have usefully distinguished several 
forms of hill climbing. 

(a) The goal is a unique maximum of a numerical valued 
descriptor on the machine s parameter space wherein points re¬ 
present the computed function. The learning machine is a simple 
optimizer. 

(b) The goal is a maximum of several local maxima, in which 
case the machine must include a ‘random’ perturbation in order 
to avoid suboptimal maxima. 

(' c ) goal is a spike in a plain’ in which case the strategy 

of the machine must be ‘hill finding’ rather than ‘hill climbing’. 
This situation is trivially dealt with by the unpracticable ex¬ 
pedient of ‘random’ search. In fact a very different machine is 
needed as argued in 2.7 and 2.8, adding a further case to the 
Minsky and Selfridge categories. 
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{d) The goal is an optimum sequential decision policy which 
can be shown to exist in several important cases (in particular 
when different values of the parameter (j) determine Markovian 
subsystems which have the property of generating well-defined 
values of the stochastic variable to be optimized). 

With the possible exception of (a) above (which constitutes 
a limiting case) the function computed by the adaptive machine 
is probabilistic in which case the relation of eqn (1) is more 
conveniently written as: 

y=x{F*) (3) 

where 

^=lli>if 11=^0/1 and( = 1,2, ...,n and 

7 = 1,2,..., m are indices of x { e X and of y$ e Y. 

Thus C selects amongst a set of n.m. conditional probability 
matrices P^. If the domain and the range of all P$ is the same 
(the state sets X and Y) the overall controller may institute a 
reinforcing strategy. In this case, if 6 0 is a given value of 6 , the 
entire machine computes conditional probabilities. 

P,i, ( y s \xi) = p(yj , 6(x, y)>9 0 \ x t ) = p (y } , C | x f ) (4) 
where the event £ occurs if and only if 6 < 0 O . 



In this case, G of (2) becomes a function that maximizes O 0 
so that the system as a whole selects a sequence j)c7 that, given 
x a X, maximizes the expectation that the value 0 (x, y) > 0 Q 
for O 0 set as high as possible. 

If (j> is discrete valued (as one assumes) then a useful iso¬ 
morphic representation of a learning machine is derivable by 
replacing the parameter variation by a selective process amongst 
computing machines or subcontrollers labelled in Figure 2 as 
Ci, C 2 ,..., C m , (j> = 1, 2, ..., m able to compute one of the m 
possibly computable functions (the sign ® implies that c is 
selected, not that c receives an input channel, denoted as 1 c|). 
Since the inputs and the outputs of the c are parallel connected, 
it is unrestrictive to impose the rule that one and only one of 
them is selected at once. It is, of course, possible to select adap¬ 
tive subcontrollers, which is tantamount to selecting a selector 
and that set from which it selects. This image emphasizes the 
idea of ‘selective amplification’ first considered by Ashby 22 and 
the related concept of ‘an hierarchy of control’. 

The selected subconfroller, C+, makes selections, when it 
computes, from y £ Y on the evidence of a sequence x c: X. On 
the other hand, B selects amongst the c e C by assigning a value 
to the index variable (j) on the basis of evidence that has been 
abstracted from the behaviour of the environment when control¬ 


led by its own selection of C. Hence the selective process is 
partitioned and the selections made by B are ‘amplified’ in the 
sense that a research director’s selective activity is ‘amplified’ 
when, instead of selecting from solutions to a set of problems, 
he selects, instead, from a set of qualified scientists who solve 
most of these problems on his behalf. Regarded as a computer, B 
computes a function G with domain the values of 0 (a, y) [an 
index set of the product set ( X , Y)] and with range the values of 
<p (an index set of the set C). Hence G is a function of highei 
logical order than F ', and B is higher in an hierarchy of control 
or organization than any subcontroller ce C. Indeed, in selecting 
a subcontroller, B selects a sign denoting the invariant that this 
subcontroller maintains. The decisions made by a learning 
machine that acts as a control mechanism are made about some 
abstract representation of the environment. 

In order to consolidate our ideas, the pioneer work of 
Uttley and Ashby is briefly considered. Uttley has stressed 
the process of abstraction and of inference about the existing 
state of affairs whereas Ashby is mostly concerned with issues 
of control and stability. These different approaches, however, 
turn out to be complementary and lead to very similar learning 
machines. 

2.3. Learning Machines Derived from Uttley s Conditional Prob¬ 
ability Machine 

Consider a set of binary variables, a *, b *, c* 9 ... which as¬ 
sume non-zero values if and only if the environment possesses 
corresponding properties, a, b, c, .... Although there is no limit 
upon the number, M , of binary variables that may appear, 
M = 3 is used for illustration. 

The environment can be classified by an abstractive network 
of AND units, as shown in Figure 3, where a particular output 
of the network, such as a • b 9 is denoted a ‘pattern’, and a ‘pat¬ 
tern’ (which is one element in a system of classification) is said 
to exist in the environment if (in the case cited) a* * b* = 1. 

The occurrence of a pattern (except the exhaustive pattern 
a- b * c ) does not exclude the presence of one or more other 
patterns (thus a* - b* = 1 does not exclude the possibility of 
a*- fr*. c * — i a i so ) t However, the network can be made to 
discriminate up to 2 M states of the binary variables if NOT 



□ = AND unit 
Figure 3 
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j ] = AND unit 
/\ = Unit delay 

Figure 4 


fir is inversely proportional to z(rj r ) = z(r) 

oc ir is proportional to z(rj r -rj i ) (8) 

These threshold units are arranged in the hierarchical struc¬ 
ture of an abstractive network, the output of any unit providing 
an input term of any unit in which it is conjoined. Thus for units 
a, b , and a • b there is the structure of Figure 5. 

The value of ftt&, given a * = 1, is the estimate of p (a-b | a) 
and the machine may infer a - b , given a if ft a b > L. The crucial 
point of Uttley’s construction is that the conditional probability 
computation depends upon a pair of variables (interpreted as 
and ft). In this connection, Maron 23 ’ 24 has recently demonstrat¬ 
ed that a variable threshold element which forms a product from 
a set of m weighted inputs can emit an output impulse (designat¬ 
ing the assumed truth of an hypothesis) in consonance with an 
optimum Bayes decision strategy. In the first place it can be 
shown that an optimum decision strategy for the yth threshold 
element given the input evidence, rj l9 ..., r) m ,j > m, is satisfied by 


units are also incorporated. The network can also respond to 
sequences of events if suitable DELAY units are added. In the 
simplest case, if one and only one of a*, Z>*, and c* is non-zero 
at once, the network of Figure 4 can recognize sequential pat¬ 
terns like 4 a after 6’ and £ b after a\ 

If the environment is deterministic, the output of such a net¬ 
work asserts the patterns which exist. On the other hand, if the 
variables a *, b* and c* convey an imperfect image of the at¬ 
tributes a , b and c, its output provides evidence on the basis of 
which inferences can be made about the patterns that exist. Thus 
a* = 1 may provide evidence that a- b is present even though, 
on this particular occasion, a* - b* = 0 (the underlying assump¬ 
tion is that the environment has stationary statistical character¬ 
istics which can be estimated by a machine that has experienced 
the environment over an appreciable interval). 

A conditional probability machine (several have been built) 
is able to make inferences on the basis of counts and ratios of 
counts of the number of occasions upon which each output of 
an abstractive network has previously been energized. The 
number of previous energizations of output r is denoted as z (r) 
= z (r* == 1) when, citing the output a-b as a typical case, the 
estimate 

converges to p(crb\ a)=p(a-b | a* = l) (5) 

in a stationary environment. Choosing a critical level of L the 
machine that embodies these requirements may infer a-b if, 
given a * = 1, p (a • b \ a) is greater than L thus 

Given a* = 1, infer a - b if p (a - b j a) > L (6) 


rjj = 1 if and only if p (rjj | rj u 


r lm) = ( r lj\rii)>~ 


(9) 



Both the ocy and the threshold fij are interpreted as variables 
dependent upon event counts that converge to p values (to main¬ 
tain correspondence with the rest of the discussion, assume that 
A’ — 1 so that = ft • rjj). If these variables are identified as 


lJ POlj) P(Vj\rii) 
and 


/*/= 


pOtj) 

P(Vj) 


where rjj implies the absence of an impulse rjj, so that p (rjj) is 
the probability that rjj = 0, then it is readily shown that (9) will 
be satisfied if an element has the transfer function 


Uttley 13 . u > 15 has built conditional probability machines 
using logarithmic scale counters and forming ratios by the sub¬ 
traction of logarithms. He points out that a threshold unit r 
with impulse inputs of amplitude r H - p u weights a ir and output 
Vr' fir, such that 7 } r is a unit impulse that occurs if and only if 

(7) 

i 

where p r is the threshold, can form the basic unit of a conditional 
probability machine providing each unit includes counters such 
that 


i = m 

nj= 1 if and only if J] afi rni >p j (10) 

i = 1 

Elements of this kind have been simulated by Maron and 
are physiologically plausible. 

Since at least a couple of variables, aij and jtij are involved 
and since a logarithmic measure of the event counts allows the 
element to decide on the basis of a difference between the thresh¬ 
old and a sum there is a close relationship between this model 
and Uttley’s variable oc, ft, system. 

Regarding Uttley’s work, an abstractive network of threshold 
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units can be regarded as anj adaptive filter having an output of 
patterns, indicated or weighted with a certain amplitude of 
signal, and an input of binary variables indicating the existence 
of attributes. After some experience of its environment, this 
filter, given the index of an attribute, provides a high amplitude 
output from those channels associated with patterns in which 
the given attribute has often been conjoined (Thus, given the 
occurrence of a , the output of a • b is high if a * b has often 
occurred). 

Defining rarity as the negative logarithm of a probability, 
Uttley considers a conditional rarity machine built in the same 
manner but from units in which eqn (8) is replaced by 

proportional to z (t] r ) 

a ir inversely proportional to z(rj r ‘r} t ) (11) 

which adapts to become a filter that given the occurrence of an 
attribute provides a high amplitude output from channels as¬ 
sociated with patterns in which the given attribute is rarely 
conjoined. (Thus, if a • b often occurs and a • c rarely occurs then, 
given a , the output of a • c is of high amplitude and the output 
of a * b is of low amplitude). 

It has been assumed that an abstractive network is speci¬ 
fied (so that the relevant patterns in the environment have already 
been determined). If this is not the case (and commonly it is not) 
it would be possible to build a complete network, with 2 M out¬ 
puts. Providing that all the relevant attributes have been indexed 
by binary variables, the relevant patterns will exist in its output. 
However, the number of elements required to build this network 
becomes gigantic for large values of M and the construction 
would be impractical. Further, in a control mechanism, the 
relevant patterns which constitute the evidence for a decision are 
normally restricted. Hence it appears necessary to start off with 
uncommitted elements which the machine connects together de¬ 
pending upon its experience of the environment by some process 
of maturation which might be called ‘learning an abstraction’. 
The first possibility is that connections develop between coin¬ 
cidentally stimulated units a and b and a unit that comes to index 
their conjunction a * b . Uttley discards this hypothesis because 
it entails the existence of a unit labelled a • b to recognize the 
a and b conjunction in the first place. The alternative hypothesis 
is an over connected network of units which differentiates and 
becomes more selective as a result of its experience. The organi¬ 
zation of a conditional rarity machine can be shown to develop 
in this manner, 



Figure 6 


Suppose there is a conditional probability (or a conditional 
rarity) machine delivering indices of the conditional probabilities 
of several states x v e X 0 denoted as p (x v \xi\ given any x% e X — X 0 
where X 0 e X is a particular subset of states entailed by a control 
procedure with a decision rule A, from X 0 into 7. This machine, 
combined with an apparatus that embodies A, is a control mech¬ 
anism. 

The environment can only be specified statistically. Hence 
the decision rule A is embodied in a probabilistic device that 
selects yeY as a function of an independent chance-like process 
that is biased by s \ \p (Ar v |^«)ll as in Figure 6. Thus the system’s 
behaviour is expressed by 

y-x(P) 

forxeX — X 0 where P= || p(jj | x,)|| = ||A [p(x ; | x,)]|| (12) 

which is consonant with eqn (3). 

The formulation is essentially unchanged if the output selects 
amongst sequences of motor activities, rather than distinct re¬ 
sponse states. 

To complete the picture, an adaptive reinforcing procedure 
must be introduced. Several mechanisms have been proposed by 
Uttley and Andrew, of which the most convenient is to adjoin 
a further state, f, which exists if and only if 6 > 0 o as in (4), to 
each conjunction in the abstractive network. Thus the machine 
computes conditional terms, which are ‘conditional upon rein¬ 
forcement’, yielding a set of values 

p(y-£\x) 

as in (4) and comment that the abstractive system shown in 
Figure 7(a) is characterized, at a particular instant, by a matrix 

P+=\\p(yt\x)\\ 

where <p is the adaptive parameter. 

Variation in the parameter (p will adequately describe the 
maturation as well as the adaptation of a probabilistic machine 


(a) 


(b) 



Figure 7. In practical systems direct 0 reinforcement is used, or, alter¬ 
natively, 0 Q is changed each finite interval by an adjustment + A0 if the 
mean value of d61 dt over an interval r exceeds 0, or — A Q, if not (a) 
and (b) are functionally identical 
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providing that it is always presented with the same statistically 
stationary environment. When the environment is non-stationary 
however, the p values are in continual flux. If the environment 
is characterized by several distributions n i9 I7 2 , there 

may be up to L different successful adaptations and the machine 
will have to re-adapt whenever i7 r , r = 1, 2,..., L is changed. 
Thus, having adapted to TI r by forming P^ r the machine may 
be presented with II r+1 when it must start to adapt once again. 
As a result of achieving the successful adaptation P^r+x it loses 
all trace of P^ r and is at no particular advantage if IJ r occurs 
subsequently. What is needed (and what may readily be provided) 
is an attention mechanism Q as in Figure 8 selecting amongst 
differently matured networks according to evidence from the 
immediate behaviour of the system. Each box M in Figure 8 
represents a complete machine of the form II in Figure 7 select¬ 
ed by Q according to the convention of Figure 2. 

Other probabilistic learning machines (Steinbuch’s 25 learn¬ 
ing matrices, a system due to Ratz and Thomas 116 , and the 
‘Eucrates 5 systems devised by Bailey, McKinnon Wood and 
Pask 26 ) closely resemble the original formulation of Uttley’s. 
They differ chiefly in emphasis (in the ‘Eucrates 5 system, for 



example, the idea of an attention mechanism is emphasized) or 
in identification (they are not, as the Uttley model is, wedded 
to a neurological hypothesis). 

2.4. Other Learning Machines Built from Networks of Elements 

There are many adaptive networks in which the coupling 
weights oiij between the elementary units are varied as a function 
of the activity of the system and an externally manipulated rein¬ 
forcement variable. Taylor 27 * 124 for example, has an artefact 
made from linear elements (the impulse rate at the output is 
proportional to the weighted sum of the impulses arriving at 
several inputs). Widrow 28 , Willis 29 and others have built net¬ 
works of threshold devices (which satisfy eqn (7) with f$ a con¬ 
stant) and Babcock 30 constructed a special purpose computer in 
which the elementary units can be variously programmed. All 
of these systems can be induced, by suitable training, to respond 
in a given fashion to patterns of input stimuli. Probably the 
largest body of data in this field is due to Rosenblatt 31 who has 
fabricated many different adaptive pattern recognition machines 
called ‘Perceptrons 5 . A typical Perceptron is shown in Figure 9. 
The response 5 units are threshold units with constant weights. 
The association 5 units are threshold devices with variable weights 
ccij, the change A oc^ depending upon the correlated activity 



Figure 9 

appearing at the input and output of theyth element and upon the 
approval of the external mentor. Rosenblatt’s original descrip¬ 
tion allowed for ‘random 5 connectivity. More specialized versions 
of this system have properties that vary from a passive logical 
filter, able to abstract a given attribute of its input to self- 
oscillatory systems within a back coupled mesh. Between these 
limits are networks that act like conditional probability mech¬ 
anisms. The idea of ‘random 5 connectivity, which appears rather 
often in the literature, is more fruitfully reinterpreted as a modi¬ 
cum of overconnection in the sense cited in 2.3 that is 
removed as a result of adaptation. In the first place the network 
is neither ‘random 5 for the reasons cited in 1.8, nor is it ‘random’ 
in the sense of being unrestrictedly variable. As Papert 32 points 
out, any m input network which was unrestrictedly variable (in 
the sense of being able to compute all 2 2m functions of m binary 
input variables) would be unrealizable for sensibly large values 
of m since no training strategy could reach the desired adaptation 
within an acceptable time. Fortunately, the issue does not arise 
in practice for many special limitations are built into the specifi¬ 
cation ; for example, threshold elements can only compute linear 
discriminating functions (if the a# are coordinates of a state 
space the values of the ay determine a linearly discriminating 
plane ^ and r\% = 1 if the input state is on one side 

of this plane and rjj = 0 if it is on the other). This case is 
examined by Papert 32 , Singleton 33 and Scott Cameron 34 who 
show that the percentage of Boolean functions that are linearly 
discriminating decreases rapidly with increasing value of m. Ivan- 
henko 35 considers similar problems in the case of a perceptron. 
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It may turn out that the class of linear discriminating func¬ 
tions, although tractable in the sense that it is a small enough 
domain for an adaptation strategy to work in a reasonable time, 
is a rather inept choice. Many other classes of Boolean function 
would be acceptable. Willis 36 , for example, proposes networks 
restricted to the computation of disjunctively decomposable 
functions of their input and it can be shown that such a network 
is separable into components, as indicated in Figure 10. It is 
important to avoid confusion between this separation of com¬ 
ponents and the hierarchical organizations considered in 2.5. 


2.5. Ashby’s Homeostat and More Elaborate Hierarchically Or¬ 
ganized Controllers 

The Homeostat is shown in Figure 11 in the form described 
by Ashby 37 (a more versatile apparatus has recently been de¬ 
veloped by Haire, Harouless, Miller and Williams 38 ) and a 
homeostat with memory has been constructed by Chichinadze 119 . 



Figure 11 


The original Homeostat consisted of four null output posi¬ 
tional servomechanisms. The positional output of any one can 
be applied (depending on a plan of interconnection) to the input 
of any other unit. Each positional output potentiometer is pro¬ 
vided with limit indicators and, in the original demonstration, 
was identified with one essential variable of an organism. Ashby 
used the apparatus to demonstrate ‘ultrastability’ which is a 
generalized version of adaptive control. Suppose that the homeo¬ 
stat has a plan P x of interconnection and is given an input (from 
some external environment) that perturbs several of the units. 
Now, given P l9 the system may or may not be stable. If not, at 
least one of the essential variable potentiometers indicates a 
limit value (which is taken to contravene the stability condition 
for the system). In this case, an overall controller (equivalent 
to Q) is required to select another plan of connection between 
the units, say P 2 . If the system is stable, given P 2 and a typical 
environmental input to perturb it, no further change occurs. 
If not, the overall controller makes a different selection. 

In the homeostat, the overall controller strategy is the most 
generalized (and also the least provident) conceivable. The ma¬ 
chine aims merely for stability, P x and P 2 being selected in an 
arbitrary fashion and independently of the state of the system. 
An adaptive controller P, C, is a specialized version of this 
paradigm. However, note that the boxes c e C do not correspond 
to the physically discriminable units in a homeostat. They cor¬ 
respond, instead, to subsystems created by an interactive cou¬ 
pling specified by P x or P 2 , between these units. The computing 


machines are well defined organizations, but they are not neces¬ 
sarily well localized physical entities. This comment has marginal 
relevance in connection with simple homeostasis but it becomes 
important when considering extensive hierarchies of control as 
in Figure 12. 



Figure 12 


MacKay 39 ’ 40 was the first to consider large, hierarchical, as¬ 
semblies of computing systems. Using the argument of 2.2. 
(which is suggested by MacKay) a mechanism A may be 
regarded as selecting amongst signs that denote the invariants 
maintained by subcontrollers b e B, the b e B may be viewed as 
selecting amongst signs associated with the ce C. Hence the 
entire learning machine is capable of abstraction (in particular 
of abstracting a goal or homeostatic criterion). Andrea 41 dis¬ 
cusses the issue in detail and Mesarovic and Banerji 117 have 
developed a theory of a self-organizing system based mainly on 
the idea of an hierarchy of control. 

If the goal is well determined and unchanging, the structure 
has little interest. There are, however, many environments to be 
controlled in which the goal is nebulous, for example, if the 
control mechanism is used in management it will probably be 
asked to maintain ‘happiness’ and ‘productivity’. Now even if 
one presupposes that a machine construction for ‘happiness’ or 
‘productivity’ maintenance can be upheld, it is still necessary to 
admit that these properties are differently interpreted upon 
different occasions, and that the level as well as the form of 
abstraction involved in their expression is variable. If the ma¬ 
chine were less flexible, control would be impossible, because 
the required invariance is specified with reference to a parameter 
which is given different ostensive definitions upon different 
occasions. 

A very similar dilemma is engendered when the environment 
to be controlled is made up from incomparable parts. In this 
case, the control problem is solved by a machine that contains 
an attention mechanism as given in 2.3. In order to achieve 
a given goal, this machine must ‘attend’ to different inputs 
upon different occasions and ‘contemplate’ different response 
alternatives. 

With reference to it can be said that any machine which 
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which includes an hierarchy of control, or of goals, can, in cer¬ 
tain circumstances, be said to contain an hierarchy of meta¬ 
languages and that any machine able to build such an hierarchy 
of control can, in the same circumstances, be said to ‘construct 5 
metalanguages. Manifestly, such a machine can build abstrac¬ 
tions and, given the provisions of 1.5 it may be said to use 
‘concepts’ 42 * 4S . 


2.6. A Self-organizing System , Localized , and Unlocalized Auto¬ 
mata 

Before considering the issues of attention and goal varia¬ 
tion in earnest, let us examine one, slightly more tractable, sys¬ 
tem. This is a special case of a so called ‘self-organizing system 5 
(it is commonly agreed that the term ‘self-organizing system 5 is 
unfortunate since the entity concerned is really a sequence of 
systems). Now according to von Foerster 44 * 45 > 46 a subsystem 
Jr (one member of this sequence) is a self-organizing system if 
the rate of change of its redundancy, R, is positive 

dR(J r )/dt>0 (13) 

The special case to be considered is a model that represents 
the data processing entity which is referred to as ‘a man 5 when 
one speaks about a ‘man 5 learning or performing a realistic skill. 



There is plenty of evidence that a conscious ‘man 5 , regarded 
as a data processer or computer, has the characteristic that in 
order to survive he must maintain a certain rate of adaptation 
(the environment must provide sufficient novelty for him to learn 
about) 44 . Another undeniable fact about a ‘man 5 is that he is, 
over any finite interval, a finite automaton with a well defined 
input called his ‘field of attention 5 (and, incidentally, a well de¬ 
fined output of responses that in some way act upon his ‘field 
of attention 5 ). In so far as the training routine or the actual skill 
occupy his attention, the field of attention, with relevant input 
states X r and output states Y r is coupled to relevant stimuli and 
responses as suggested in Figure 13. This condition would not 
be satisfied if the man were bored or inattentive. In order to 
satisfy the basic requirement that a certain rate of adaptation 
is maintained, the environment must be capable of offering sub¬ 
sets of stimuli and responses (X u Y r ) that have a statistical rela¬ 
tionship II r which the man can learn. Where (X r , Y r ) cz X, Y 
is the product the set of relevant stimuli and response states. 

This primitive image of man can be modelled with a set of 
the conditional probability machines given 2.3 and character¬ 



Figure 14. Selection of n 2 , P^ 2 , is shown 


ized by matrices P^ r as in eqn (3) and having input states X r 
and output states Y r . The environment is characterized by a set 
of matrices TI r (which may also be embodied in a machine as 
suggested in Figure 14). It will be convenient to assume that P^ QV 
has equal p entries. Calling the rth coupled subsystem consisting 
of ‘man 5 and his immediate environment J r , it is obvious that 
the adaptation rate condition is equivalent to eqn (13). In this 
model one can insist that J r will maximize its chance of survival, 
say by computing R ( J r ) as an index of adaptation and, from eqn 
(4), making 


e r =R(j r )= i- 


H(J r ) H(X r ,Y r ) 

H (, J r ) max H (X n Y r ) max 


(14) 


and since H ( X r , Y r ) max is invariant for a given coupling, one 
has 6 r « —H(X r , Y r ). Thus, if eqn (14) is substituted in eqn 
(4) and if the values of the adaptive parameter <j> converge, 
-> 4>n • • • T one obtains 

d6 r /dt « dJR (J r )/dt > 0 (15) 

for </) 4= T. Wattanabe 47 , for example, has considered the dy¬ 
namics of systems like this which must adapt and points out 
that learning behaviour as well as learning models satisfy 


R (/,)| fas -H(J r )\t = a(t-t 0 )e~ bt 


where a and b are positive constants and where t> t o ^0 is the 
instant of observation. J r , however, is obviously an unstable or 
ephemeral self-organising system. Since the reinforcement vari¬ 
able, 6 , is increased by any increase in the regularity of J r be¬ 
haviour, the system will approach a stable (though possibly 
dynamic) equilibrium characterized by 

P Tr = stochastic inverse (LI r ) (16) 


However, if eqn (16) pertains, J r is fully adapted, hence, at this 
point, 

d0 r /dtndR(J r )ldt = 0 (17) 


contravening eqn (13) so that J r is no longer a self-organizing 
system (it can be demonstrated that although a ‘forgetting 5 fa¬ 
cility can delay the instant at which eqn (17) applies, ‘forgetting 5 
does not avoid the ultimate instability). When this occurs, a real 
‘man 5 will ‘change his attention 5 . To comprehend this fact, any 
model (cf. Figures 14 and 17) must include the transformation, 
say 2, embodied in a mechanism A with domain a set of M or 
more subsystems J r , J r e /, and with range J. Let A receive 
an instruction, whenever eqn (17) is true, to select X (J r ) = J r+1 
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which consists of a conditional probability machine character¬ 
ized by P^, r +i coupled to an environment characterized by ll r + 
For an initial selection of subsystem J 0 and for M iterations of 
this process the minimal constraints 


yield a sequence 


A (J r ) e J and A (J r ) # J r 


r = M 


•/*= u [r(j 0 J] 

r = 1 


( 18 ) 


such that d R (J r )/dt > 0 for all J r e J* £ /. Thus J* is a self¬ 
organizing system. 

This model serves as a very primitive image of ‘man’ provid¬ 
ing (a) that the adaptation takes place with reference to some 
goal (like performing a given skill), not just arbitrarily (increas¬ 
ing R is tantamount to performing any skill), and (b) that 
although, in the limiting case, A may act when eqn (17) is true, 
it is also possible to replace this instruction with an internally 
or externally produced 'attention directing’ signal. 

A more basic and interesting distinction between this model 
and reality is that, in fact, the set J is not denumerable. J is de¬ 
fined (and consequently renders the disjunction of eqn (18) 
legitimate) in terms of some property of /* which leads to as¬ 
sertions like 'these systems all represent the behaviour of a man’, 
or ‘of an animal’ (which, since they refer to the systems rather 
than the behaviours, are metalinguistic statements). 

Typically, this situation comes about when there is a re¬ 
productive process capable of creating and replicating machines 
characterized by P<f>r which survive if and only if, when coupled 
to form systems like J r , dO/dt > 0. In this case, if a sequence of 
M systems does survive, the transformation A is interpretable as 
a rule of evolution that, in a specified environment, gives rise to 
/*, and the members J r of J* are characterized by the property 
that they are produced by this evolutionary rule. 

The point leads us to a different, ‘evolutionary’, kind of learn¬ 
ing machine. The previous learning machines stem from the 
familiar model of a localized and finite automaton (a finite auto¬ 
maton with well defined input states and output states). The 
model for evolutionary systems is a population of reproducing 
automata. 

G T (G) 




—s- Functionally specified components 
—Couplings 

Figure 15 

Suitable constructions have been developed and discussed by 
Burke 48 , von Neumann 49 , and Loefgren 50 * 51 . In Loefgren’s 
development of von Neumann’s work, it is pointed out that 
a dual representation of these automata is commonly and per¬ 
haps necessarily adopted. 

(a) As a developing connection graph (with which there is an 
associated state graph). This method has been adopted by 


Rashevky 52 , Rosen 53 , and others. A transformation, T in 
Figure 75, maps graph G into graph T(G). Often enough T can¬ 
not be readily expressed in an analytic form. 

(b) As a configuration on a ‘tesselation’ (a ‘tesselation’ is an 
infinite plane of cells, i, y, each of which can assume one of a 
finite number of states u e U % 9 u e Uj , amongst which, for each 
value of /, at least one state is a null state denoted as u 0 e Uj). 
Entry into a null state is interpreted as the disappearance of 
some aspect of an automaton, which is a configuration of the 
states of certain neighbouring cells, and any transition from the 
null state implies the creation of some feature of an automaton. 
The transition rule, Z, depends, for the /th cell, upon u e Ui and 
the state of neighbouring cells. Hence Z is a mapping which if i 
and j are neighbours is of the form Z; (Uu Uj) (Ui). 

If an automaton is localized its (a) image is a structure of 
functionally defined components and its (b) image is a connected 
configuration of other-than-null states. If an automaton is un¬ 
localized, in particular if it ‘evolves’, its (a) image is often diffi¬ 
cult to interpret because of an ingrained confusion between 
organizations and objects, whereas its (b) image is a replicating 
population of distinct configurations. 

It is worth noticing that: 

(1) In his discussion of ‘error correction’, Loefgren shows 
that in a real universe, where an automaton receives permanent 
structural perturbations as well as irrelevant inputs, no localized 
automaton can have an infinite life span but that some un¬ 
localized automata can have an infinite life span. This distinction 
demarcates the set of evolutionary learning systems. It is impor¬ 
tant, however, to avoid confusion between this and the well- 
known distinction of minimal automata and automata that 
are deliberately built with structural redundancy in order to 
effect error correction. 

(2) As emphasized in 1.2, the simulation of a learning 
machine entails the simulation of its normal environment. If 
learning is interpreted as the reproduction of an organization 
(which is, for example, the view adopted by Wiener) the environ¬ 
ment can be said to mediate the reproductive process. In fact, 
this interpretation (or some equivalent) is necessary in order to 
avoid Rosen’s 54 paradox (which is a special case of Russel’s 
paradox), but the term ‘environment’ implies an internal ‘envi¬ 
ronment’ such as a brain or some other physical embodiment in 
which the evolutionary process can be realized. It can be called, 
unspecifically, the medium in which evolution takes place 55 . 

Because of the confusion between organizations and objects 
it is often difficult to specify the medium in the (a) image, but 
the (b) image shows a population in (or an organization that is 
a property of) the most abstract possible representation of a 
medium, namely a tesselation. This is a straightforward picture 
and, for most purposes, is preferable. 

Discussion of abstract systems that learn, in the evolutionary 
sense, is always related to the dual pictures of (a) and (b) but 
some contributors to this field, such as Masano Toda 56 , Heinz 
von Foerster 61 , Caianiello 58 , and Pask 59 > 60 , have been differently 
motivated and oriented. A far less abstract version of a medium 
is used, akin to a close coupled, but real environment. However, 
it remains true that the evolving organization is a property of 
the medium rather than something that exists in its own right 
and a couple of points are worth emphasis. In the first place, 
the evolutionary rule, Z, is a property of the other than minimal 
specification of the medium thus illustrating the material 


403 






GORDON PASK 


dependence cited in 1.5. Next, the medium in which learn¬ 
ing systems can evolve is highly constrained, but the constraints 
do not refer directly to the computations manifest in the learning 
behaviour. They exist in order to ensure that evolution of any 
system can take place. 

2.7. Different Realizations of Learning Machines in Evolving 
Systems 

Selfridge 61 has devised an hierarchically structured pro¬ 
gramme called ‘Pandemonium’ that can be used either for re¬ 
cognition or for control. The ‘Demons’ in a Pandemonium are 
computing routines and are isomorphic with possibly adaptive 
subcontrollers, as shown in Figure 16. The lowest order demons, 
in contact with the environment, recognize or control prede¬ 
termined features, indicating their achievement by signals con¬ 
veyed to higher order demons that assign weights to these sig¬ 
nals and transmit a linearly weighted evaluation of the lowest 
demons, performance to an overall controller demon which is 
criticized by some external mentor. 



The system resembles a Perceptron in so far as the middle 
demons perform a linear weighting and in so far as there is some 
external mentor. It differs a great deal in respect to the lowest 
demons, which in Pandemonium are highly specific feature re¬ 
cognizers or subcontrollers. A more important difference lies, 
however, in the development of the system, which is not revealed 
by the instantaneous picture. Whereas adaptation in a perceptron 
entails only an adjustment of weights, the demons in a Pande¬ 
monium must be capable of evolution. The designer is supposed 
to be prudent in his choice of features, but not omniscient. Con¬ 
sequently, subcontrollers that meet with common disapproval, 
evidenced by uniformly low weight assignments, must be discard¬ 
ed and others created to take their place. If a subcontroller in 
the organization of a Pandemonium is identified with a species 
of automata, the organization could be realized by a fairly simple 
evolutionary process. Members of a population of automata 
compete for some commodity (food or money) that is available 
in short supply and which allows them to persist and to repro¬ 
duce (this tacitly assumes a cost, in terms of food or money, 
for maintaining the fabric of an automaton and a similar cost 
for its replication). The supply of food or money is sufficient to 
maintain, on average, a number, n , of automata, which (depend¬ 


ing somewhat upon the distribution of automata in different 
species) can sustain the activity of the required number of sub¬ 
controllers. Since more than n automata are created there is 
competition and some automata fail to survive (indeed, those 
that survive will be members of species favoured by the higher 
order components that act, presumably, upon evolutionary 
rules). 

Given some kind of variation, there will be a selection of 
favoured variants. The issue is, what principle of selection is to 
be used? Purely chance variation would be pointless, for the set 
of possibilities is too large, but two mechanisms are possible, 
namely: 

(1) Recombination. Instead of discarding an unpopular de¬ 
mon and creating a novel demon, the elements of a partially 
successful demon are recombined in a fresh arrangement. Hence, 
amongst other things, the adaptations of the original demon are 
not completely lost. 

(2) Co-operative interaction whereby the original demons are 
coupled into co-operative entities which are reproduced, by some 
different and possibly evolved mechanism, as a whole. 

Although a model of this kind is easy to describe and parts 
of it have been successfully simulated, there is a tendency to 
consider media more like brains than the environments of simple 
automata. Pringle 62 , suggested that hierarchies of control could 
evolve in a large multiple mode oscillator due to the non-linear 
coupling between stable oscillatory modes. Such a system is 
conveniently realized by a large network of self-exciting thresh¬ 
old elements, with malleable coupling. The existence of a stable 
mode imprints a pattern upon this medium which, in turn, in¬ 
duces the stable mode or filters its components from uncorrelated 
excitation (in a stable system there will be a many to many cor¬ 
respondence between patterns and dynamic components). The 
existence of a stable mode acts as an informational ‘metabolism’ 
that maintains the constraints which are required in order that 
the network shall compute a specified function of its external 
stimulation (hence modes of oscillation are, in this case, the sub¬ 
controllers of demons). Networks like Beurle’s simulation 9 ’ 10 , 
mentioned in 1.2 are admirable media for this purpose and 
Beurle has examined the self-excited as well as transmissive 
characteristics of this system. Amongst others, Farley and 
Clark 63 have simulated very large assemblages of somewhat 
simplified non-linear elements. Taylor’s paper, at the present 
meeting, deals with this area. 

From a theoretical viewpoint, Wiener 64 has recently propos¬ 
ed a potentially quantitive theory of learning and has tentatively 
applied it to data from cortical activity. As a generalization of 
our previous comments about oscillatory modes, replication 
(which is viewed as the basic mechanism of learning) is re¬ 
presented by the distortion free propagation of a wave of activity 
in a medium of non-linear elements. (The comments on filtering 
are generalized in terms of Wiener filters.) Further development 
of this work should lead to a precise and comprehensive image 
of the evolutionary mechanism that underlies learning 65 . 

2.8. Problem Solving , Communication and Strategy in Linguistic 
Environments 

As suggested in 1.5, there are many situations in which 
the control engineer is required to build learning machines 
that interact with a symbolic environment. Often these situations 
are obvious enough; for example, the related issues of language 
translation and data retrieval certainly entail learning (at any 
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rate in large systems) and it seems unlikely that the learning of 
sign sequence correlations is sufficient (the machine must learn 
the language concerned). The same is true for the whole of 
'artificial intelligence’. Other situations are not so obviously 
symbolic in character. Tustin 66 has demonstrated that many 
facets of the economy may be predicted and perhaps controlled 
within a model that is derived from Laplace transformations of 
economic data. As a result, the design of a control mechanism 
appears to involve an application of conventional methods. 
However, there are discontinuities, due to invention, for ex¬ 
ample, which prove intractable within the framework of Tustin s 
model. One can, of course, neglect these or, failing that, one 
can interpolate a guess, but it seems at least possible that a 
machine capable of symbolic interaction might guess solutions 
to these problems, better than oneself, but by using ones own 
methods which are (by hypothesis) to discern relevant signs in 
an underlying cultural language and to interpret them as indices 
of change (this, incidentally, will involve feedback perturbations 
of the controlled system and a machine capability for learning 
as well as manipulating the language). 

Another situation involving symbolic interaction is man 
machine symbiosis, for example, in problem solving (the term 
‘symbiosis’ is used to distinguish the present relation between the 
man and the machine from the more familiar relation in which 
a machine is used as a problem solving tool). In symbiosis, the 
aim is man machine co-operation. The machine should sug¬ 
gest hypotheses and proofs to the man, as well as carrying out 
his instructions. In order to do this, it must develop a common 
language (so that the man machine interaction has the logical 
calibre of a conversation). As a result, in a genuinely symbiotic 
system, it would be impossible to tell whether the man or the 
machine was responsible for the solutions achieved. 

Very little work has been done, so far, in symbiotic problem 
solving but some results are available for comparable symbiotic 
systems used for training (the control of learning) and mental 
testing (stabilizing the man machine interaction to maximize 
the information available regarding the values of specified para¬ 
meters). 

A few specific mechanisms are now discussed in an attempt 
to illustrate some of the chief difficulties of the field. 

In order to talk about signs, it is first necessary to specify an 
object language L 0 (which is used by the system concerned) in 
terms of an observer’s metalanguage L *. As Cherry 67 has pointed 
out, this expedient is a formal prerequisite for any kind of ex¬ 
perimentation (but it is tacitly assumed and taken for granted 
when the experimental technique is well known and the L* inter¬ 
pretation of L 0 signs is invariant). If one also wishes to assert 
that an organism or a machine manipulates signs (in particular, 
that it learns about signs or acquires concepts) one must either: 

(a) Be in a position to embed symbolic representations with¬ 
in the machine (trivially by building them into it and not entirely 
trivially by the ham-fisted reinforcement of an adaptive network 
so that it recognizes the symbols determined by its external 
mentor), or 

( b ) Understand a common environment (by knowing ‘essen¬ 
tial variables’, for example, so that, in the absence of rein¬ 
forcement it can be taken for granted that the organism or 
machine will learn to use signs denoting the critical value of 
these ‘essential variables’ if it survives. Failing this, it would 
encounter the critical condition rather than a sign for it). 


Returning again to 1.5, the lact that not all abstrac¬ 
tions can be generalized is tantamount to the fact that not all 
signs act as symbols for a given machine and its environment. 
To cite a picturesque but defensible analogy, the population of 
automata in an evolutionary learning system must (if they inter¬ 
act co-operatively as in 2.7) possess an ‘internal language 
with symbols of its own. Generalization is possible if this ‘inter¬ 
nal’ language can be translated into the external language and 
vice versa). 

Having established L 0 and represented it in L* a control 
mechanism can be regarded as a problem solving device. The 
perturbations to which this controller responds correctively pose 
problems (certain equivalence classes of xeX denote prob¬ 
lems and certain y e Y that index the controller s asymptotic 
response denote solutions). The controller becomes a problem 
solver (an adaptive controller becomes able to solve a particular 
problem) and the functions computed by the fully adapted 
machine specify a decision rule. A largely equivalent formulation 
images the machine as a game player when certain y e Y are its 
moves, certain xeX are the moves of another player, 6 (x, y) 
is the payoff function of the game over the outcome set X, 7, 
and the computed function determines strategies, y (as describ¬ 
ed, the game is played in its extensive form). 

An abstractive or hierarchical learning machine is capable of 
quite elaborate problem solving and game playing since it can 
learn to solve a class of problems by constructing descriptions 
of problem solution. An hierarchical structure can embody as 
many machine metalanguages L l9 L a ,... (describing and manip¬ 
ulating L 0 ) as there are levels of control, and evolutionary sys¬ 
tems are able to build L r , r = 1, 2,... (which are, in some sense, 
interpretable as conceptual languages). 

The issue of 1.5 reappears, however, namely ‘in what sense 
are these levels of control conceptual levels?’ Providing there 
are no overtones of mentation, any abstractive learning machine 
can have and some can build ‘concepts’, but in order to 
remove the inverted commas, it is necessary to consider the dis¬ 
course between this machine and other machines or other men. 

A pair or more of learning machines can interact at several 
levels of discourse, L 0 , L u ... either through couplings establish¬ 
ed between different levels of control or, due to similarity of 
construction, by inferring the L 2 , L 2 , ... implication of expres¬ 
sions in L 0 . George 68 » 69 has experimented with multi-machine 
computer simulations and has examined the linguistic require¬ 
ments entailed in the successful conduct of non-zero sum, partial¬ 
ly co-operative, games and certain sequential games. The entire 
experiment is explicated, of course, in L*. 

On the other hand, in connection with ‘artificial intelligence’, 
it is not only necessary to describe the problem solving in terms 
of L*, but also to interact, personally, with the machine. 

A typical man machine system is proposed by Edwards 70 . 
The job of constructing a Baye’s estimate of the probability of 
an hypothesis, conditional upon certain data 


p(Hyp | Data) = 


p (Data 1 Hyp) • p (Hyp) 
p (Data) 


is ideally divided between a man and a machine. There is evi¬ 
dence to suggest that a man can be trained to accurately deter¬ 
mine p (Data | Hyp) which a machine cannot readily do (especial¬ 
ly if much of the data is irrelevant or if the hypothesis is tenta¬ 
tive) providing he receives a guide from present values of 
p (Hyp | Data). On the other hand a man is a notoriously in- 
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accurate calculator and in Edward’s system, the Bayes’ calcu¬ 
lation is conducted by a machine. 

The extreme case of man machine interaction, conversational 
man machine symbiosis, is exemplified by the adaptive teaching 
system in Figure 17 (these systems have been used to instruct a 
number of industrial skills including perceptual motor skills, 
like card punching and intellectual skills like fault detection on 
the basis of imperfect evidence) 71 ” 74 . 

Broadly, the machine acts like a personal instructor who 
changes the training routine to suit an image of the student, 
derived from continual observation, by posing novel problems 
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Figure 17 


when the student is bored and simplifying the material or co¬ 
operating if the displayed data is too difficult. Precisely, the 
system is a stochastic optimizer based upon the model given 
in 2.6. The machine selects subenvironments, characterized 
by the different matrices fl r , with an operation Q analogous to 
the operation X (in the ‘student’ part of the model). Adjoining 
the plausible condition that each of the adaptive ‘automata’ 
(that momentarily represent the student) must receive intelligible 
problems to solve, and also interpreting 6 r as an index of 
relevant regularity derived from a success measure in such a way 
that A6 r is an index of learning rate in the rth subenvironment 
(or for the rth subskill) it is possible to specify a strategy. 

Now Figure 17 represents a feasible system if there is enough 
information about L 0 to determine a ‘structured skill’ (in partic¬ 
ular to reduce the skill to sets of problems x* e X* <= X r con¬ 
cerned with the rth subskill wherin if, for each r, there is a ‘sim¬ 
plifying’ procedure that renders any x* e X* a more intelligible 
simplified problem, denoted by a stimulus x e X r —X*). If so, it 
can be shown that a pair of strategies Cj and C 2 will" maximize 
the rate of directed self-organization in the system J* of the 
model of 2.6, namely: 

Q given that X p is selected, hence J p> vary the simplification 
of an arbitrary sequence of x selected from Xp to minimize the 


degree of simplification and to maximize A 0 V . If this proves 
impossible in J p inform A and institute C 2 . 

C 2 Let A select X r , hence J r , such that the expected value of 
M is maximized. Proceed with C t . 

There is some evidence to indicate that C L and C 2 are, in a 
certain sense, ‘optimal’ strategies that maximize learning rate, 
but for most skills a much more elaborate hierarchical adaptive 
mechanism is needed to realize C ± and C 2 . Note that although 
the selections made by strategy C t are expressions in L 0 , the 
selections made by A, embodying strategy C 2 , are expressions in 
a machine metalanguage, L x say, since they instruct the student 
to direct his attention to a different subskill 75 . 

In Figure 18 a similar adaptive device is used to minimize the 
error rate in an inspection skill. The adaptive machine in 
Figure 18 injects ‘false’ signals into the inspector’s display, senses 
whether he misses the signal, and adjusts the injected signal 
distribution to minimize this error rate. All ‘false’ signals are 
automatically excluded from the system output. 

These machines have the status of catalysts that control an 
evolutionary process in the student’s brain. They could be ex¬ 
tended (by providing a capability for building an hierarchy 
■f'l’ • • • of metalanguages) so that this process became increas¬ 

ingly exteriorized (the machine could act as a medium, compa¬ 
rable to the brain, in so far as it manipulated ‘concepts’ like the 
student, but due to co-operation there is a possibility of a man 
machine compromise about the form of ‘concept’). 

If the man is removed from this extended system, it becomes 
an undiluted artificial intelligence’. The ‘induction principles’ 
demanded by Church 76 , Markoff 77 , and others in an intelligent 
system must stem from the evolutionary rules that constrain the 
machine. Given that these are specified rightly, the ‘artificial 
intelligence’ will solve problems as man solves them. One method 
of embedding these rules might be to allow a learning machine 
to mature in contact with a group of men and a common lin¬ 
guistic environment. The other possibility is to apply constraints 
upon the machine which may be 

(a) Heuristics determining classes of solution or classes of 
algorithms. 

(b) Cost functions, determining how much the machine has 
to pay in terms of a necessary and limited commodity, to main¬ 
tain different structures or to apply different algorithms. 
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Some features of the better-known intelligent learning ma¬ 
chines are now briefly outlined: 

(1) One of the less elaborate systems is Manfred Kochen’s 78 
problem solver which applies various search strategies in a situ¬ 
ation proposed by Bruner, Goodnow and Austin 79 in their 
studies of problem solving and learning. Its behaviour compares 
reasonably with human behaviour in the restricted experimental 
environment of these studies but the system is not intended to 
do more than this. This learning machine is an adaptive device 
that modifies the parameter, <£, of the function set defined by its 
strategies. 

(2) The hierarchy of control (in a symbolic identification of 
languages) is isomorphic with the structure of a list programme. 
Hence problem solving machines like Newall, Shaw, and 
Simon’s G.P.S. 80-82 are abstractive learning machines. Variation 
of the hierarchical structure takes place as a function of the 
experience gained by applying the listed tests. Learning has at 
least a couple of different forms, (a) Learning to apply a test 
which entails the development of a local metric over the choice 
tree (equivalent to adaptation at a given level as in Fiegenbaum 
and Samuel’s device 118 ), (b) Learning to order tests and modify 
lists, which amounts to changing the hierarchical structure. 

Minsky 83,84 has particularly emphasized the role of heuristics 
or broad principles of problem solving that are built into a 
machine and guide it in selecting the algorithm it applies to its 
working data and to its own structure. Unless these are built in¬ 
to the device there seems no hope that any sort of order will ap¬ 
pear. The heuristic constitutes an assertion of how a species of 
learning machine will look at its environment. 

(3) Together with Minsky, Marzocco, Travis 85 , and others 
at S.D.C. take the very reasonable view that the most important 
heuristic determines a principle of restricted generalization 
whereby a given machine learns to apply a ‘generalized form 
of a previously successful algorithm when it is subsequently pre¬ 
sented with a ‘similar’ problem. The crux of the learning process 
is to assign a connotation to the words in inverted commas. 

This is also true of Amarel’s 86 ‘theorem proving’ machine 
that ‘learns to learn’ in a limited domain (it proves theorems 
that are mappings from a finite product set of elements into one 
coordinate of this set). Even here the search process (which 
entails developing choice trees and building up lists of descrip¬ 
tions of these procedures) is impracticable unless it is somehow 
restricted. Amarel urges (a) the need for a variety of descriptive 
metalanguages L 0 , L t ,... in any intelligent learning machine ( ) 
the need for some kind of economic control over the search 
process. Apart from obvious measures like informational value 
that can be used to characterize the result of a test, the machine 
must have to pay for making tests and maintaining the test 
systems in terms of a restricted commodity like the food or 
money discourse in 2.7. A similar point is made by Solomonoff . 

(4) Maron 23 , points out that the basic problem of artificial 
intelligence is choice. The logical rules that are built into a 
machine and the probabilistic bias it may acquire by learning, 
determines only what it may not do (the set of alternatives it is 
allowed to contemplate and those it may not contemplate). Noth¬ 
ing is said about what it should do within the contemplated set 
(an instruction to ‘throw a dice’ indicates our indifference but 
the act is realized by a specific and often inept ‘decision proce¬ 
dure’). A machine that genuinely learns to solve problems must 
learn to choose and is likely to develop preferences and oddities. 


3. The Mechanism in which Learning Machines are Realized 
3.1. Brain Analogies 

(1) The brain is probably the best immediately available guide 
to how the various constraints should be realized by the control 
engineer. This realization is the concern of Bionics or Biological 
Cybernetics. Many of the hoary contentions of this field, like 
how ‘random’ or how ‘specific’ a brain may be, are loosing their 
edge. Nowadays, few people subscribe to the doctrine of ‘ran¬ 
dom’ networks and few people imagine that the brain is a large 
‘telephone’ system. There remain a number of less extremist 
biologically oriented design principles for learning machines that 
appear to have more than polemic value. These must be applied 
with the recognition that other than biological brains may have 
entirely different optimal designs. 

(2) Brains do (and one suspects that learning machines must) 
have a well defined overall plan. TheL 0 informational interface* 
between the organism and its environment is characterized, at 
the input, by an hierarchically arranged filter that extracts per¬ 
ceptual attributes. At the output, there are hierarchically con¬ 
trolled motor repertoires that give rise to sequences of coherent 
actions such as ‘speech’ or ‘eating’ each of which can be regarded 
as a whole. Between these specialized regions, there are other 
more or less organized entities concerned with integration and 
various forms of control. Probably the most comprehensive 
cybernetic model is provided by Anohkin 88 . 

Within such a broad plan the brain has well defined func¬ 
tional units such as the various distinct memory systems, dis¬ 
cussed, for example, by Brown 89 and Barbizet 90 . 

(3) Very specific filtering networks have been constructed by 
von Foerster 91 and by Harmon 92 . This work has been stimulat¬ 
ed and guided by the physiological investigations of Lettvin, 
Matturana, McCulloch and Pitts 93 , of Hubei and Wiesel 94 
of Reichart 95 and others. Novikoff 96 has outlined principles for 
designing attribute filters on the basis of integral geometry. An 
axiomatic specification of a pattern is given by Lerner and 
Vashnik 123 . 

At a more macroscopic level, Lerner 121 has proposed reflex 
analysing mechanisms which suggest and have been embodied 
in machine organizations. Finally, it can be argued that there is an 
information theoretical analogy between the filters of a learning 
machine and certain forms of communication and computation 
channels. Agalides 98 considers the matter at a microscopic level 
and Broadbent’s" model (which is interpretable as a psycho¬ 
logical hypothesis) refers to the macrostructure of a learning 
machine. 

(4) Some of the filters in a brain are adaptive and some are 
predictive. On the basics of this analogy, Barlow and Donald¬ 
son 100 have built a highly specific coding system which adapts 
to minimize the redundancy in an input sequence which could 
be plausibly derived from an attribute filter. Uttley’s 101 two 
variable conditional probability theory is interpretable in terms 
of a neurological mechanism through an identification between 
eqns (9) and (10) of 2.3 and some physiological process. 

* Notes. The L x or L 2 description interface is normally different. 
A man or any learning machine has a definite location with refer¬ 
ence to a given descriptional language. If this language is not specified 
he or it has an undetermined boundary. Man is bounded in one 
way as a social being and in another way as a food eating creature. 
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He is, at the moment, testing this hypothesis and the view that 
dendritic fields in the brain provide the connectivity of a predic¬ 
tive network. Almost all of the adaptive filters have some claim 
to biological precedent. Thus Rosenblatt’s Perceptron was in¬ 
spired by Hebb’s learning theory and assemblies of the kind that 
Taylor and Steinbuch have built are consonant either with 
Hebb’s view 1 2 or more specific theories such as those of Milner 103 . 

In brains, the hierarchical structure of control is partly 
apparent in the anatomy. Bishop 104 points out that the Broad 
Pattern (which is anatomically defined) has evolutionary origins. 

Braynes, Napalkov and Sechvinsky 105 , and Napalkov 97 , 
have developed a particularly elegant description of conditioned 
reflexes in terms of algorithms. In many animals it is possible 
to exhibit the hierarchy of control as an interaction between 
different levels of algorithms which can, of course, be rigorously 
specified as a relation between normal algorithms in the sense 
of Markoff. The hierarchy of algorithms describes a structure 
of reflexive chains which have, at any moment, a definite loca¬ 
tion, but there is no reason why the description should always 
refer to the same physical structure. Hence the hierarchy of 
algorithms is an organization which sometimes may and some¬ 
times may not reflect an anatomical hierarchy. These authors 
have built several machines to illustrate their own ideas and the 
obviously related concepts of the response hierarchy biologists 
such as Tinbergen 107 . One of the most comprehensive hier¬ 
archical learning systems is described by Andrea 41 . 

(5) The brain is a parallel computing mechanism and it has 
a great deal of structural and functional redundancy. Work such 
as Cowan’s 1J8 on error correction in computing networks leads 
to an image (the design of a typical error correcting, stable, 
network) which is also highly redundant and which closely re¬ 
sembles the connectivity of a brain. In this kind of system, the 
same function may be mediated by many different components 
in many different locations. The brain is many physical realiza¬ 
tions of a computing system rather than a single realization (a 
brain is functionally similar to a computing system—it bears 
little resemblance to the object called a computer). The minimal 
components are not always well defined and the attributes of 
physical events that count as signals are rarely well defined (thus 
in some parts of a brain, neurone groups act as the minimal sub¬ 
systems, in other parts, single neurones, and in others the com¬ 
ponents of a neurone. Similarly, ‘signals’ may be impulses or 
the phase relations between impulses or the mean rate of im¬ 
pulses, or analogue variables that are not directly related to im¬ 
pulses). It is not unreasonable to specify this kind of micro¬ 
scopic redundancy of function in the components of any learn¬ 
ing machine. Certain obvious advantages are obtained. The 
trouble is that the idea of a ‘component’ and the familiar dis¬ 
tinction between signals and objects that modify these signals is 
lost. 

Finally, there is a redundancy at the level of macroscopic 
properties. It is probably pointless to ask what in a brain medi¬ 
ates its memory. An indefinitely large number of physical mech¬ 
anisms might (and in the sense of being history dependent) must 
act in this capacity. The real issue is to discover which are re¬ 
levant to the computations that interest us. Thus a brain can 
have memory in terms of excited loops of neurones, in terms of 
a distribution of facilitation that favours the propagation of a 
particular wave of activity, in terms of specific macromolecular 
changes at the synaptic interface and possibly in terms of local 
and specific changes in messenger R.N.A. There is no reason to 


suppose that any one of these is responsible for any facet of 
memory and their relative importance probably depends upon 
the state of the brain. Given the component specification sug¬ 
gested a moment ago, similar properties may be expected in a 
learning machine. 

(6) The brain includes an attention mechanism, chiefly em¬ 
bodied in the reticular formation which, according to McCul¬ 
loch and Kilmer 109 can be regarded as an input and output 
transforming system with parameters controlled by the higher 
centres. It determines the level in an hierarchy of control at 
which a feedback loop, involving the data occupying the brain’s 
attention, will be completed. Ideally this is the level of abstrac¬ 
tion suited to a given type of data. It has already been argued, 
on entirely different grounds, that an attention mechanism is 
needed in any competent learning machine. 

(7) The fact that a brain is a collection of closely coupled 
evolutionary fragments, does not appear to be accidental, nor 
is the fact that the ontogeny of a brain recapitulates the phyto¬ 
geny of the brain of a species as the system undergoes the process 
of maturation. Such manifestations as imprinting (that certain 
behaviours are acquired by the organism if and only if a specific 
state of its environment occurs at a specific point in its develop¬ 
ment) are a predictable consequence of maturation and unless 
an alternative is suggested, any realistic learning machine must 
also undergo a process of maturation as proposed in 2.8 in its 
normal environment. 

In this connection. Young’s investigation 122 of the octopus 
has recently revealed that the brain of this animal is an hier¬ 
archically organized homeostatic system in which the hierarchi¬ 
cal levels are related to the sensory and motor systems especially 
characteristic of distinct evolutionary stages (touch, visual dis¬ 
tance receptors, and visual pattern systems). The octopus is 
peculiar in that the functions of its brain are localized (and, 
hence, are readily discriminated) whereas in most animals the 
functions are distributed in a way which obscures the overall 
plan. However, one may hypothesize that the organization ex¬ 
posed by Young’s work persists, even when its embodiment is 
hard to discern as it is in a mammalian brain. 

(8) There are regions in a brain that could, very readily, act 
as the medium in which dynamic organizations evolve. Relatively 
undifferentiated parts of the cerebral cortex display an activity 
like Beurle’s networks and although it is possible that evolu¬ 
tionary processes, in the sense given in 2.8, occur in almost 
any biological system, it is tempting to suppose that these parts 
are specially developed as a ‘medium’. There is plenty of evidence 
that evolutionary processes, in the sense given in 2.8, go 
on in brains, that they underlie the learning behaviour of the 
associated organism and serve to extrapolate its biological evo¬ 
lution into the field of language and social interaction. It has 
been argued, in less flowery terms, that much the same comments 
may apply to learning machines. 

3.2. Non-bio logic a l Embodiment 

Since useful learning machines are certainly large, their com¬ 
ponents (with the reservation of (5) above upon using this word) 
must be small. There seems no reason why one or another of 
the microminiaturization techniques (such as the techniques dis¬ 
cussed by Shoulders) cannot be used to fabricate the perceptual 
filters and motor hierarchies of (2), (3) and (4) above in conven¬ 
tional circuitry. The fabrication need not be perfect, since the 
system is stable against crass structural disturbances. These 
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methods are not able to realize a competent version of matura¬ 
tion [required by (7) above] or of a medium [as in (8)]. So far 
as maturation is concerned, the constructing device for etching 
or embedding the circuit would have to be controlled as a func¬ 
tion of the interaction between the brain it is producing and the 
normal environment of this brain and although this is not impos¬ 
sible, the expedient seems, at first sight, to be unduly cumber¬ 
some. 

Various attempts have been made, chiefly by MacKay and 
in my own laboratory, to ‘grow’ connectivity in a network using 
metallic threads or dendrites that develop by electrodeposition 
in a form determined by the activity induced through the con¬ 
nections they instantaneously determine. Whereas MacKay 110 
has brought to competence a method of ‘growing’ dendrites that 
act as connective elements, our own work yielded a very crude 
but somewhat wider application of this process. Briefly, it was 
possible to construct, from dendrites and capacitors alone, a 
realized network of Crane’s 111 * 112 ‘neuristors’ (a ‘neuristor’ is 
an active transmission line, of which a nerve fibre is a special 
case. Crane has shown that suitably coupled networks of neu¬ 
ristors are able to compute any Boolean or Probabilistic func¬ 
tion. Their realization, using Lilley’s passivated wires, is con¬ 
sidered by Stewart 113 ). In the present system, instable dendrites 
subserved the non-linear switching action of a neuristor whereas 
stable connective dendrites acted as the coupling and charging 
impedances. If the system is chosen so that the deposition is 
partly reversible, the dendrites have a reproductive character¬ 
istic and competitive and co-operative interaction takes place 
between members of a population of developing dendrites 4,26,59 . 
Since, as Feldman 114 points out, a neuristor system is ideally 
suited to act as a medium for evolving dynamic organizations, the 
dendrite realization of a neuristor network appears particularly 
attractive and it may be possible to use this system now that 
more elegant procedures have been developed. 

The basic criticism is that the size is too large. Ideally, the 
active transmission lines should be large macromolecules and 
Bowman 115 argues that certain polymers could be used for this 
purpose. To complete the specification, it is necessary to transmit 
signals along polymer ‘neuristors’ whilst the polymerization is 
in progress, and to use these signals through a local catalytic 
action to control the further extension of the ‘neuristor’ network. 
Physical chemists seem willing to countenance the possibility of 
controlled catalysis, at least in principle. 
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Invariance of Sampled-data and Adaptive Sampled-data Systems 

V. M. KUNTSEVICH and Yu. V. KREMENTULO 


Summary 

In the report it is shown that the invariance conditions can be used 
successfully in usual continuous control systems and also in sampled- 
data control systems. This is especially important in adaptive impulse 
systems. In the first section of the report fundamentals of invariance 
theory of combined sampled-data systems are considered. For such 
systems the invariance conditions of the controlled variable were 
obtained both for sampling intervals and for any period of time. 

Possibility of a significant improvement of adaptive systems based 
on the invariance theory is investigated. The impulse extremal system 
of the incremental type with a test signal is considered. It is shown 
that by introducing a certain change in the structure of the adaptive 
system and according to the choice of parameters, a complete elimi¬ 
nation of the system error due to a definite group of disturbances 
which displace the extremum points along the axis of controlled vari¬ 
able, is possible. In this case the errors due to these disturbances are 
eliminated and the region of stability is considerably expanded. 

Sommaire 

Dans ce rapport on montre que les conditions d’invariance definies 
dans les systemes continus peuvent etre appliquees avec succes aux 
systemes de commande echantillonnes. Ce point est particulierement 
important dans les systemes adaptatifs impulsionnels. La premiere 
partie de ce rapport traite des bases fondamentales duplication de la 
theorie de l’invariance aux systemes echantillonnes mixtes; les con¬ 
ditions d’invariance des variables commandees sont determinees 
durant les intervalles d’6chantillonage et pour toute periode de temps. 

On examine ensuite la possibility d’ameliorer les systemes adapta¬ 
tifs sur la base de la theorie de l’invariance. On considere un systeme 
extrdmalisant impulsionnel du type a increments avec signal d’essai. 
On montre qu’en modifiant la structure du systeme adapta if et en 
effectuant un certain choix de parametres, on obtient une dlimination 
complete de l’erreur due au deplacement de Fextremum, ce ddplace- 
ment de Fextremum etant du lui-meme a une categorie de pertur¬ 
bations bien definies. Le domaine de stability se trouve considerable- 
ment ytendu. 

Zusammenfassung 

Der Aufsatz zeigt, daB sich die Invarianzbedingungen (Kompensation 
von Storeinfiiissen) sowohl in den ublichen kontinuierlichen Regel- 
systemen als auch in Abtastregelsystemen erfolgreich verwendet 
lassen. Dies ist besonders fur selbsteinstellende Abtastsysteme (Im- 
pulssysteme) wichtig. Zunachst gilt die Betrachtung den Grundlagen 
der Invarianztheorie kombinierter Abtastsysteme. Die Invarianz¬ 
bedingungen fur die RegelgroBe solcher Systeme werden sowohl fur 
die Abtastperiode als auch fur jeden anderen Zeitabschnitt abgeleitet. 

Es wird untersucht, ob eine wesentliche Verbesserung der selbst- 
einstellenden Regelsysteme auf Grund der Invarianztheorie moglich 
ist. Ein Extremalwert-Abtastregelsystem mit schrittweiser Naherung 
durch ein Testsignal wird betrachtet. Es zeigt sich, daB durch eine 
bestimmte Strukturanderung des selbsteinstellenden Systems und auf 
Grund der Wahl der Parameter eine vollige Ausschaltung des System- 
fehlers moglich ist, sofern der Fehler einer bestimmten Klasse von 
Storungen angehort, die die Extremstellen entlang der Achse der 
RegelgroBe verschieben. In diesem Fall werden die auf derartigen 
Storungen beruhenden Fehler ausgeschaltet und der Stabilitatsbereich 
betrachtlich erweitert. 


One of the important scientific trends in the theory of automatic 
control is the theory of the construction of systems on the basis 
of compensation of the influence of disturbances, or the theory 
of invariance of the controlled value. 

As is known, however, the invariance theory was recently 
used extensively only for ordinary continuous control systems 1 ” 7 . 
Attempts were made in a number of works 8 ” 15 to extend the 
general principles of this theory to sampled-data control 
systems, but there has not yet been any full and systematized 
statement of the invariance theory for such systems. The above 
statement relates in a still greater degree to adaptive systems in 
general and sampled-data systems of this type in particular. 
Since adaptive systems are a special type of non-linear systems, 
then, as will be shown below, the introduction of compounding 
disturbance links makes it possible not only to improve the 
quality of systems when compensating the influence of disturb¬ 
ances, but also to extend the stability region of these systems. 

The authors consider that the main aim of their paper is to 
demonstrate the fact that the sampled-data system analysis and 
synthesis methods expounded below can serve as the basis for 
the construction of control systems with considerably greater 
accuracy than existing systems. 

Henceforward the following constraints and assumptions 
will be accepted: (a) synchronous sampled-data systems with 
amplitude modulation are considered; (b) the sampling period 
T is constant; (c) the pulse element is ideal; (cl) the equations are 
written in deviations; and (e) initial conditions are zero. 

Since sampled-data systems of fairly complex structure will 
be considered, the consideration will begin with the method of 
solving the equations of multiloop sampled-data systems. 

Sampled-data Systems 

Multiloop Sampled-data Systems Equations 

A number of works 16 ” 23 have been concerned with the 
compilation and solution of the equations of sampled-data 
systems. The solution of the equations of multiloop sampled- 
data systems is given in the most general and convenient form 
by Burshtein 17 . The method suggested below has features in 
common with Burshtein’s method, but allows one to avoid a 
number of intermediate operations and to simplify the calcula¬ 
tions. 

In the most general form the equation for the /rth coordinate 
of a multiloop sampled-data system can be written thus: 

n n Ifct 

x k (s) = y ^i(s)x ; (s) + y y *?(*)&*«(*) 

i— 1 i= 1 j= 1 

m m P ki 

+ I S*(s)F,(5) + £ E C' kij Ff (z) C kij (s) (1) 

i— 1 £ = 1 jf = 1 
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where x lci are the coordinates of the system, F { the external 
disturbances, n, m the number of selected coordinates and 
external disturbances respectively, l ki , P lci the number of parallel 
links (pulse-continuous) between the coordinate x k and x t and 
the coordinate x h and the external disturbance respectively; 
W, b , b f , c , c and R are the corresponding transfer functions, 
shown in Figure 7, which depicts part of a multiloop sampled- 
data system (A;th node). 



If one takes into consideration the additional coordinates: 


where 


Y a i j(s)x J (s)+ Y a' 1 j(s)xJ(z) = A 1 (s) 

j=i 

N N 

Y a NJ (s)Xj(s)+ Y a' NJ (s)x%z) = A N (s') 

j =i J= i 

N=n+ Y E hi 

k = 1 i= 1 


( 3 ) 


is the full amount of coordinates of the system (including the 
additional ones). 


40) 


m 

= - 

_i= 1 


m Pki 


WF|W + E Z C' ki j F* (z) C kij (s) 

i=lj=l 


a kj (s)~W ki (s) ; a kk (s) = W kk (s)-1; etc. 


and are numbered in accordance with (2). 

System (3) formally contains TV equations with 2 TVunknowns, 
Xj (s) and x* (z). As in ref. 17, the terms containing z transforms 
of the coordinates will be transferred to the right-hand side. 
The resulting system will be solved relative to the arbitrary 
coordinate Xj ( s ). This gives: 


Xj(s) 


__Axj(s) 
A(s) 


where 


A (s) = 


a il 0), ■■ 

., a 1N (s) 

a Nl (Os *■ 

., a NN (s) 


(4) 

(5) 


is the common determinant of a purely continuous system. 


Eqn (6) 

Determinant (6) may be presented in the form: 


^11 In (0 ^1 (0 + 0 ) ( 2 ) 

b'lnl (s)x„(s) 

^n + lu + ’"h (« -1)+ l 0) 


Knh„ 0) X„ (s)= x„ +hi + ... (s), etc. 

then the equations of the multiloop sampled-data system can 
be given in an ordered form: 


Eqn (7) U 


The first of the determinants entering into (7) will be denoted 
by A a * (s), and the remainder by A j x * ( s ). Bearing in mind the 
notation adopted: 


Xj(s) = 


40) 

A(s) 


Y ** 0 ) 


k = l 



( 8 ) 


* Eqn (6) 

4,00 = 

flu («)»•• 

,,a u _ 1 (s);J 1 (s)- Y a' lj (s)x*(z);a 
j- 1 

1J + I (0s •• 

‘? a lN (0 



N 





a Ni (Os *' 

.,a Nj . 1 (s)-A N (s) - Y a Nj 0) x* (z); a NJ + x (s),.. 
j- 1 

•s (0 

t Eqn (7) 



(^h •••, ^lj+ 1 CO 5 ^1 (0? ^1j + 1 (0? • 

a ljV (0 



4, 

0) = 

a Ni (0? 

Ujyjy ( S) 




N 

(0? •••5 ^ij -1 (0? (Os • 

..,a 1N (s) 



’s 

JL 

l x*(z) 
= 1 

%1 (0s •••? a IVj- 1 (0 s a m c (0? 1 (0s 

a NN (s) 
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Subjecting (8) to a z transform and cancelling identical terms, 
the following relation is obtained: 


**(*) 


i+(^f 


(*) 


#)VE^)(^f 

(fe# j) 


00 ; 

(9) 


The initial system of equations is: 

<p(s)+0- (s) + 0 = (5) X (s) 

- ^ (s) (s) + 1 ? (s) - FF y/i (s) [i (s) ■+ 0 

= »; a ( 5)A(5 )+w;,(s)^( 5) (U) 


Thus the initial system (3) can immediately be transferred 
into a full system by equations of type (9). The full system of 
equations of a multiloop sampled-data system has the form: 

Z flij(s) Xj(s)+ £ a' u (s)x*(z)=^ 1 (s) 
i=1 j =1 


- WV (S) <p(s)+0+n(s)- W lw (s) n* (z) 

= ^(s)A(s)+^(s)^(s) 

In accordance with the method expounded above, this 
system is made into a full one by the deficient equation: 


N 


E 

t — 1 


N 

a nj 0) Xj (s) + E a Nj 0) x* (z) = Xjy (s) 


j = i 



xf(z) 


JV 


+ Z 





( 12 ) 


Henceforward, only programme and servosystems will be 
considered; hence, in (11), X (s) — 0. 

From (11) and (12) one can easily find an expression for the 
controlled coordinate in which one is interested. 


( 10 ) 


<p(s) = K z (s)\l/(s) 


N- 1 


E 



(z)x* (z) + 




When writing the determinants forming part of (10), the 
following symbolization is accepted. The upper index shows 
which column of the common determinant A is subject to 
substitution, while the lower index indicates substitution by 
coefficients for particular variables. Thus, A k x*j means that the 
Ath column of the common determinant A is to be replaced 
by coefficients at the /th discrete coordinate. 

System (10) can be solved, relative to the coordinates of 
interest, by ordinary, algebraic methods. Sampled-data systems 
with various types of link will now be considered. 


+ - 


K 3 (s) 


1-K*(z)~K 3 K*(z) 


{1(K 5 + K 2 KM*(z)} (13) 


where 


y /a - ( s ) ( 5 ) . y (A — 

2(} l-W,„ u (s)W„MY i{ ) ~\-W„ 


(s)»r w (5) 


' W (»)^(s)’“ 3w 1 -w„{8)w„{8y 

K 5 (s)=W dc (s)+W 0I1 (s)W^( S )i 


K 6 (s) = W (nv ( s ) W vc (s) + W Dft ( s ) (5); 

K 1 (s)=W Vfl (s)W llv (s) 


Conditions of Absolute Invariance. The condition of absolute 
invariance for servo and programme systems is: 


Sampled-data Systems with Continuous Compounding Links 

An automatic control system with one pulse element, which 
can be described by a system of three linear equations with 
constant coefficients, is studied. The block diagram of the system 
is given in Figure 2, which also shows the transfer functions 
of both the main loop and the additional links. 



Figure 2 . Block diagram of combined control system 
I: plant 


cp (s) = K 2 (s) i// (s) 

K 3 (s) 


+- 


or 


1 -K*(z)-K 3 K*(z) 


m 5 +K 2 K 6 )tT(z)} = <Ks) 


-s(s) = K' 2 (s)f(s) 

I *3 ft) 

1 -K*(z)-K 3 Kt(z) 


(14a) 

{i(K s +K 2 KM*(z)} = 0 

(14b) 


where e (s) = ip (s) - <p (s) is the system error K' z (s) = K 2 (s)-1. 

The basic differences between the conditions of invariance 
for continuous and sampled-data systems is emphasized. While 
in continuous systems the conditions of absolute invariance do 
not depend on the form of ip, and are determined only by the 
parameters of the components of the system, in the sampled-data 
system under consideration, these conditions (14) essentially 
depend on the form of the input signal ip. 

It can be shown that the condition of absolute invariance 
physically signifies the equality to zero of the sum of the indi¬ 
vidual components of the coordinate £ produced both as a result 
of the direct effect ip upon the system, and also on account of the 
effect via the additional (compounding) links. 
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Invariance Conditions for Discrete Moments of Time. The 
invariance conditions (14) were obtained from the requirement 
of the equality to zero of coordinate s at any moments of time. 
One may impose a less rigid requirement—the equality to zero 
of s at the sampling instants, i.e., 

c[nT]= 0 (15) 

The conditions under which (15) is satisfied are called 
‘conditions of invariance for discrete moments of time’. If (14) 
is subjected to a z transform, then it seems that the problem is 
solved. However, it is easy to show that the invariance conditions 
for discrete moments of time as well, will depend upon y. 

An attempt is made to obtain the conditions, independent 
of yj. Both parts of (14 b) are multiplied by 

K 5 (s) + K 2 (s)K 6 (s) 

K' 2 (s) 

and then subjected to a z transform. 

(i Y; w , /W ..{kA*,, w)*oo 
\K ’ 2 7 Z ^ (Z \K'J \-K*(z)-K 3 Kt(z) 


Condition (20) is satisfied if [(K 5 + K 6 )/K 2 r ] + K 6 contains 
proportional components or components with a pure time lag. 
From (20) and (17) can be found the transfer functions of 
continuous compounding links. 

Sampled-data Systems with Discrete Compounding Links 

A brief examination will be made of the properties of a 
typical sampled-data servo-system, the block diagram of 
which is given in Figure 3. The expression of the system 
error e is: 

£ * ( z \=— M EeuM ( z ) (21) 

() i +w v *(z)K\(- z r 




h Vi r 


is obtained, where / (s) = K 5 (s) + K 2 (s) K 6 (s). 

By equating the right-hand side of (16) to zero, the following 
invariance conditions are obtained for discrete moments of time: 


1 -jK?(z) + 


K 3 (K 5 + K 6 ) * 

K' 2 J 


The conditions of absolute invariance for a similar con¬ 
tinuous system (i.e., a system having the same structure) can be 
given in the form: 

1 _ Kl (s) + (s) (s) + giM l=0 (18) 

K'i ( s ) 

If (18) is subjected to a z transform, eqn(17) is obtained, 
i. e., the introduction of a pulse element into an absolutely invariant 
continuous system does not impair the conditions of invariance 
for discrete moments of time for the so-called \ fictitious coordinate 

£(p (s) = - —— e (s) 

' K' 2 (s) 

As shown by Krementulo 10 from the equality to zero of 
e (p \nT], there still does not follow the equality to zero of s \nT\. 
The additional conditions will be given, under which s [nT] = 0, 
and does not depend on the form of ip. (14b) is subjected to a 
z transform, and then 1 - K*(z), found from (17), is substituted: 


*r _ K * (z) 

8 Z) ^ } fe(A 5 +i^6) l* (: 
x{K 5 r(z) + l(K2 + l)KjT(z)} 

The additional condition: 

I^ +K *) £W ]V(^r + 


C Z) + K 3 K* 6 (Z ) 


k 5 +k 6 


r 2 '!' ( z )= 



Figure 3. Block diagram of servosystems: ( a) with direct link with 
respect to assignment; (b) with indirect link with respect to assignment 

The condition of invariance at discrete moments of time is: 

_* ^ 1 /'•"i'l'v 


W£(z)- 




(z)K’ 2 r(z) 

( 20 ) 


In the general case, (z) and (z) are the ratio of 
polynomials according to the positive powers of z, the power 
of the numerator being less than that of the denominator. 
Since IV,*,, (z) must be inverse to W^(z), then it cannot 
be physically realized (advancing components are required 
for this). 

It is important to note that the introduction of the link 
W; v (z) and the satisfaction of the invariance condition (22) 
do not alter the characteristic equation of the system: 

K*(z)P*(z) + Kt(z)Q*(z) = 0 ; 

/ Kp (z) _ j y*( z y P ( Z ^ = W* (z)j (23) 

\KUz) W °‘ {Z) ’Q*(z) J 

and therefore do not influence the stability of the system. 

Examples were given by Kuntsevich 12 to show that even in 
those cases when W* p (z), obtained from condition (22) cannot 
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be realized, provided it is selected in a particular way, it is 
possible to increase considerably the accuracy of a samp lin g 
servosystem. 

When for any reasons it is inconvenient or impossible to 
introduce the compounding link W*y, (z), one may introduce 
into the system additional links, equivalent to the direct com- 
pounding link (z). Eqn (21) can be brought to the form: 


8*(z) = 


1 + W*(z)W*(.z) 




1 + W*(z)W? tl (z) 


[£*(z) + <p*(z)] 


(24) 


It is not difficult to see that (24) is met by the diagram 
shown in Figure 3 (b). 

If (22) is satisfied, then the condition of absolute invariance 
has the form: 

'J'(s) _w w (s) 

nz) w* t ( Z ) ( j 

The latter equality can be satisfied only in some particular 
cases, and, as shown by Krementulo 11 , requires the inclusion 
of advancing components if xp [0] = 0. 


Sampled-data Systems With Pulse-continuous Compounding Links 


In this section a servosystem will be used as an example to 
show that when pulse-continuous links are used it is in principle 
possible to achieve absolute invariance in a combined sampled- 
data system. 


Assume that the block diagram is predetermined, i. e., 
C 5 ) an< ^ W E(p (s) are known. A compounding link 
with respect to the input signal \j/ W My} (s) is introduced to 
improve the dynamic properties. The transfer function of this 
link has to be determined. 

The expression for the system error is: 


It can be seen that for both the sampled-data and the com 
tinuous system the compounding link has one and the same 
structure and consists of identical components. The difference 
lies in the fact that in an absolutely invariant sampled-data 
system some of the components are connected via additional 
pulse elements operating synchronously and in phase with the 
main one. What has already been said also holds in the case 
when real pulse elements are used. 



(b) 

Figure 4. Block diagram of structural scheme of combined servosystem 


Extremal Sampled-data Systems 

Systems without Compounding Links 


-«(s)“[WW(s)FF w (s)-l]^r(s) 

:z)+W «*'W m 

Having equated e ( s ) to zero the condition of invariance of 
the system is obtained from which the transfer function of the 
compounding link can be determined: 

(27) 

The signal of the compounding link v 1 (s) equals: 

• (s)= KJs) +w ° c (s) (z) ~ }f/ * (z) J ( 28 ) 

This signal can be realized with the aid of the scheme shown 
m Figure 4 (b). In a similar continuous system, the compounding 
link with respect to ip, chosen from the conditions of absolute 
invariance, equals: 

( s )=^ ( s ) + W ve (s) [W e(f (s) - !] ( 29 ) 


Today a large number of extremal sampled-data systems of 
various types are known, which have been studied by many 
scientists. But certain specific features of these systems remain 
unexplained. An analysis will be made of the known extremal 
sampled-data systems on the basis of full and precise equations 
of dynamics of only one system which, as was shown in (29), 
provides the best tracking quality with continuous drift of the 
extremum, and whose properties are at the same time closest 
to those of a hypothetical system measuring the position of the 
extremum point without any errors. 

As in most works, the controlled plant with extremal 
characteristics will be considered to be one which consists of a 
linear inertial component and an inertia-less component with 
extremal characteristics. 

The equation of the non-linear component, taking into 
account the action of two kinds of disturbances (or two com¬ 
ponents of one and the same disturbance), which displace the 
extremum point, will be written in the form: 

<p=-a 3 (x + i/,) 2 +A (30) 

where cp is the index of the extremum, and ip, X are disturbances 
of an arbitrary kind, inaccessible to direct measurement by 
virtue of the conditions of the problem. Let the remaining 
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equations of the extremal system* (see Figure 5) in the absence 
of the components shown in Figure 5 by the dotted line, be: 

x(s) = W xM (s)M(s) (31) 

where 

M~n + m (31a) 

m(s)=W nw (s)V*(z) (32) 

where z 

V (z)=a' M jj- (32a) 

v(s)=W llu (s)u*(z) (33) 

y^Acp^.i-iy (34) 

u*(z)=W*(z)y\z) (35) 



Figure 5. Block diagram of a difference-type sampled-data extremal 
system with indirect compounding link 
I: plant; 1: multiplying unit; 2: memory element 

Here (31) is the equation of the linear part of the plant, 
(32) the equation of the modulation circuit, (34) the equation 
of a controller with synchronous detector, (35) the equation 
of the correcting elements, (33) the equation of the servo¬ 
motor and x, p, (p , u and y the controlled coordinates. 

Henceforward it is taken that the dynamic properties of the 
plant and the slope x 3 of the extremal characteristic are constant 
or quasi-constant. 

* Since the system under review is non-linear, then strictly speak¬ 
ing, neither the ordinary nor the discrete Laplace transform is applic¬ 
able to it. Therefore the final results will be obtained with the aid of 
a set of non-linear difference equations. To simplify things, the 
Laplace transform will only be used in application to the linear 
components. 


ADAPTIVE SAMPLED-DATA SYSTEMS 

The error of the system is denoted as: 

e=p+\j/ (36) 

and also the notations are introduced 

x* (z) =n'*(z) + m r * (z) (37) 

where 

y'*(z)=W xM W*(z)u*(z) (37a) 

m'*(z)=W mv W; M (z)V* (z) (37b) 

On the basis of (37b) and (32, 32 a), the modulating effect 
m' n , scaled to the input of the non-linear element, can be 
represented in the form 

m' = a M cos nn = a M { — 1)” (38) 

where om is determined from the particular solution of the 
difference equation 

a M (-iT=a' M W xM W mo (E)(-rr (39) 

which is obtained following the replacement of (32) by the 

corresponding difference equation. 

Solving jointly (30), (36), (37) and (38) gives 

y n = — 2 a M oc 3 ( e n 4- i) + (— 1)" — a 3 {el — x ) (— l) n 

(40) 

From (40) it can be seen that the signal on the output of the 
component (34), apart from the useful component proportional 
to the error, also contains additional terms, one of which 
AA n ~ x (— l) n reflects the influence of the disturbance A n9 and 
the third term shows that the measurement of the position of the 
system relative to the extremum point is not ideal. 

Further replacing (35)—(37) by their corresponding 
difference equations, and solving them jointly with (40) and 
(37 a), the equation of the dynamics of the system is obtained 
in the form of a non-linear difference equation with time- 
varying coefficients 

[2 a M oc 3 W(E) (E + l) + E ] e n -a 3 W ( E ) [e„ 2 + , - e 2 n ) cos mi] 

= \j / n+ 1 - W (E) \_AA n cos 7m] 

where W (F) = W xM W„ U (E) W uy (E ) (41) 

As was shown by Kuntsevich 29 ’ 30 , the non-linear eqn (41) 
has the peculiarity that for a particular correlation between the 
system parameters and the speed of variation of disturbances 
\J/ n9 A n the stability of the system is impaired, whereas analysis 
of the linearized equation obtained from (41), disregarding the 
non-linear terms (as done by Chang 25 , Van-Neis 26 and Ivakh- 
nenko 27 ) does not permit one to detect this phenomenon. 
Therefore the feasibility of constructing an adaptive system, the 
error of which would be invariant in relation to ip n , X n9 acquires 
particular interest, since it involves not only the improvement of 
the quality of the system, but also the increasing, of its stability 
margin. 

Invariance of Extremal Control Systems with Indirect Com¬ 
pounding Links 

Since, by virtue of the conditions of the problem, the 
possibility of direct measurement of the signals ip and A is 
excluded, the possibility will be considered of using indirect 
compounding links with respect to ip and A similar to those 
considered above. 
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Consideration will first be given to the possibility of attaining 
invariance of system error at discrete moments of time, relative 
to yj n *. 

From (36), (37), and (37 a) and also from Figure 5, it 
follows that 

iA*(z) = e*(z)-/*(z) (42) 

or r(z)=e*(z)-W xM n*(: z) (42a) 

For the construction of the correcting link with respect to 
ip in accordance with (42 a), the variable ju' n can be obtained 
with the aid of a model of the linear part of the controlled plant 
(see Figure 5*). A signal proportional to e n (or, more strictly, 
containing e n ) can be obtained at the output of an additional 
synchronous detector (see the part of Figure 5 outlined by 
a broken line), the equation of which is: 


y'n =<?„(-!)" (43) 

Solving (30), (36) and (43) jointly, gives 


y'n=- 2 a M a 3 e n - a 3 (e 2 + ^)(-1)"+A„ (-1)" (44) 

For filtering the parasitic quasi-periodic terms of signal 
(44) at the output of the detector in the network in Figure 5, 
a low-frequency filter is provided. 

Taking this into account, the signal on the output of the 
additional control loop is written in the form 

w n ~ D(E) (45) 

where 

D(E)=-2a M ^w;(E)W K (E) 


Omitting the intermediate operations, the equation of the 
dynamics of the system in Figure 5, with an additional control 
loop, is obtained, on the basis of the equations cited above and 
also eqn (45), in the form 

[2 a M a 3 W (E) (E + l)+£] e„-a 3 W ( E ) [(e 2 + 1 - e 2 ) cos nn\ 

= [1 — 2 a M cc 3 W xM W m (E) W;(E) W K (E)],p n + 1 
— W (E ) A A n cos nn (46) 

By equating to zero the operator comultiplier for yj in the 
right-hand side of (46), an expression is obtained of the impulse 
transfer function Wr (£), which ensures the invariance of the 
system from yj n at discrete moments of time 


W k (E) = 


1_ 1 

2 a MOC 3 W;(E)W(E) 


(47) 


From (46) it can be seen that the satisfaction of the con¬ 
ditions of invariance (47), and the presence of the filter in the 
compounding-link network (as distinct from the filter in the 
main network of the controller), do not alter the form and 
coefficients of the left-hand side of the equation of the dynamics 
of the system, i. e., do not directly influence the stability of the 
system. 


Wr* (z). An example is given in the Appendix of the method 
of selection of the coefficients of the transfer function Wr * (z). 

In deriving the conditions of invariance (47), the quasi- 
periodic non-linear terms in (44) were disregarded in order to 
simplify the investigation. As follows from the example in the 
Appendix (see also Figure 6), the influence of these terms is in 
fact small. 514 

A brief examination will now be made of the possibility 
of minimization (or complete elimination) of the system error 
due to A. From the equation of the system dynamics (46) and 
(40), it follows that for the predetermined structure the possi¬ 
bility of constructing a correcting link with respect to A(/) in a 



Figure 6. Transients of extremal system for yj n = fin, A X n = 0 

I. in system (54) with compounding link for satisfaction of con¬ 
dition (50); (oc l0 c 2 = 0-4; ot 3 = 1; d x = 0-4; d 2 = 0-8; p = 3-5) 
II: in system (55) (ditto, but d x = d 2 = 0*4; f = 2) 

III: in system (55) (ditto, but for /? = 3*5) 


similar way as with respect to yj, without constructing an analogue 
of the non-linear component, is excluded. By virtue of this, 
with the structure adopted, only methods of minimizing the 
influence of A ( t ) can be considered. One such method, based on 
the selection of the corresponding function W* y (z) was con¬ 
sidered by Chang 25 , Van-Neis 26 and Ivakhnenko 27 . The results 
obtained by Tou 24 may also be used here. 


When the required transfer function W K * (z) is physically 
unrealizable, then, as for ordinary servosystems, a considerable 
improvement of accuracy (increasing the degree of astatism) 
can be achieved by appropriate selection of the transfer function 

* It is noted that in contrast to ordinary servosystems, in which 
the input signal may also contain a no se w, izh has to be suppressed 
as effectively as possible, the task of an extremal system in all cases is 
the complete performance of signal yj. 


Appendix 

Example —In Figure 5 let 

* The system in Figure 5 was checked experimentally on an elec¬ 
tronic analogue by A. A. Tunik, and the check confirmed the effective¬ 
ness of the introduction of indirect correction 31 . 
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INVARIANCE OF SAMPLED-DATA AND ADAPTIVE SAMPLED-DATA SYSTEMS 


to which there corresponds 

7 * / \ a i a 2 (1 df)z 


w xM w*m- 


and further, let 


Wl(z)= 


(z—l)(z—d t ) 

Bt(z) 


B* ( z ) 


where £* (z) and -B.* (z) are polynomials of z, d 1 =e~ T,r ‘. 
It will be taken that 

1—e -sT I 

w'; on¬ 

to which there corresponds 

l-d 2 . 


t 2 s + 1 


no)= 


•d,’ 


(d; 


_ A -r/ T2- 


) 


It is not difficult to see that in the given case the impulse 
transfer function Wj£ (z), as determined from (47), which is 
required for attainment of the conditions of invariance, is 
physically unrealizable, and only the approximate satisfaction 
of the conditions of invariance can be considered; by virtue of 
this, W'l (z) will be sought in the form of the series 

(48) 

Denoting the left-hand side of equation (46) by L (E)e n in order 
to abbreviate the notation, one can write it for A 2 n — 0 in the 
given example, bearing in mind (48), in the form: 


L (E) e n = EB 2 (£){ —2 a M a x a 2 a 3 (1 - df) (1 - d 2 ) 


x\_C 1 A\l/ n + C 2 A 2 il/ n -i' J r ... +C K A K \j/ n - K+1 2 

+ A^ n + [(l-d 1 ) + (l-d 2 ftA 2 xl/ n +(l-dJ(l~d 2 )}Aij/ n 


Provided 


Ci = 


1 

2a M a 1 a 2 a 3 


(49) 

(50) 


the error from the first difference y n is eliminated, since, when 
this is satisfied, the equation of the system adopts the form 


of the correcting link is impossible in the given example, Q = 0 
will be taken for i > 3. 

For quantitive evaluation of the quasi-periodic terms in (46), 
which have not been taken into account, in Figure 6 the transient 
in an extremal system is plotted, taking into account these 
terms for tp n = fin, A X n — 0 for eqn (46). 

For the transfer function of the components cited in the 
example under consideration and for W* y z = 1, the precise 
equation of the dynamics of the system has the form: 

A 0 e n+3 +A 1 e n+2 + A 2 e n+1 +A 3 e n 

- a, (1 -d x ) [e 2 + 2 - e 2 n+1 + d 2 (e„\ t - e 2 )] (-1)" 

- a, (1 - d t ) (1 - d 2 ) [e 2 +1 + e 2 + 2 a 2 ,] (-1)"=0 (54) 

where 

A 0 = 1; A ± — 2 a M <Xz (l-d 1 )-(l^d i + d 2 ); 


A 2 ~ 2 a M a s (l~d 1 )(l—d 2 )+d 1 A-d 2 + d 1 d 2 ; 


A 3 = — d±d 2 — 2 a^oc^ (1 dfi)d 2 , ocjr — <Xi<x 2 cc 3 


Here, for comparison, the transient processes in an extremal 
system without correcting link with respect to ip n have been 
plotted, in which W xM (s) and W„ u (s) are the same as given 
above, and a low-frequency filter with transfer function 


W 9 (s) = 


1- 



1 

T 3 S+ 1 


is included into the main extremal-control network. Tz transform 
of W <p (s) is 

r/ 

where d 3 = e T,r \ 

Bearing this remark in mind, for the given case, the equation 
of the dynamics (41) of the system adopts the form 


A' 0 e n+3 + A' 1 e n+2 + A 2 e n+1 + A 3 e n 
— ccj(l — df) \_e n + 2 — e n+ i + d 3 (e n+ 1 — £„)] (•— i) 
= A 3 xj/ n+1 + [(1 — d t ) + (1 — d 3 y] A 2 ij/ n+ i 


L(E)e n 

= EB 2 (£){ —2 a M cL x a 2 a 3 (1 - di) (1 - d 2 ) l_C 2 A 2 xj/ n ^ 1 + ... 

+ C K A K ij/ n ^ K+ i] + A 3 i// n + (2 — d ± — d 2 )} A 2 \j/ n (51) 

Further taking 

r __ (2 — d l — d 2 ) _ ( 52 ) 

2 2 a M a 1 a 2 a 3 (1 - d t ) (1 - d 2 ) 

and bearing in mind that 

(51) can be rewritten in the form 

L(E)e n 

= EB 2 (E) { — 2ctj^cciOc 2 cc 3 (l — d^)(1 d 2 )\^C 3 A \j/ n -. 2 A- ... 

+ C K A K xl/ n - K+ 1 ] + A 3 xj/ n — C 2 }A Vn- i ($ J ) 

from which it will be seen that, irrespective of the coefficients 
W* y (z) the error is eliminated from the second difference \j/ n . 
Since further increasing of the degree of astatism on account 


+ (l-d 1 )(l-d 3 )A^ n + 1 (55) 

where 

Aq = 1; A\ = 2 ci^fOc^ (1 — df) — (1 + di -h d $), 

A 2 = 2a M cc I (l — d t )(l — d 3 ) + d 1 + d 3 + d 1 d 3 ; 

A 3 — — 2 cij^oc^ (1 — df)d 3 did 3i oc^^ocict 2^3 

As can be seen from the curves in Figure 6, an increase in fi 
(the rate of drift of the extremum) leads to the loss of the 
stability of the system (55). Thus the introduction of compound¬ 
ing links with respect to y) n not only improves the quality of 
the system, but also preserves its stability, thus extending the 
sphere of application of extremal systems to the case of high 
extremum drift rates. 
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DISCUSSION 


G. Axelby, Westinghouse Electric Corporation, Linthicum, Md., U.S.A. 

Would the author please comment on the possible application of the 
system described in his paper. 

Yu. V. Krementulo, in reply 

The method of improving the quality of sampled-data systems, which 
is based on the compounding link, can, of course, be practically reali¬ 
zed if we can build the model of the plant. This is possible in several 
cases, and I can give two examples. The first relates to the problem of 
measurement of the rate of rolling by means of correlation. The second 
relates to the problem of water refining. In this case the dynamic 
model of the plant permits the use of the extremum controller without 
test signals 1 . 

Reference 

1 Vasiljev, V. I. Automatika (1962) 

E. Jury, University of California, Berkeley, California, U.S.A. 

This paper presents a general and thorough study of the invariance 
problem in sampled-data systems, based on the assumption expounded 
by the authors. I would like to mention two points. 

(1) The introduction of the indirect compounding link is a useful 
artifice to improve on the invariance of the system. However this 
can also be achieved by introducing additional components as well as 
with a direct link. Does the indirect compounding link represent the 
minimum number of components or could it be further simplified 
without much affecting the system operation ? 

(2) I would mention that by varying the sampling rate as a function 


of the error or other signals of the system, one could also achieve the 
objectives introduced by the authors. I have studied some work along 
these lines in connection with pulse-frequency modulated feedback 
systems where certain favourable results have been achieved. A short 
discussion 1 of PFM (pulse frequency modulated) systems has been 
published and an extensive discussion of such problems has been 
recently presented. 

Reference 

Jury, E. I. On sampling aspects in feedback control systems, 
Regelungstechnik, March, 1963 


Yu. V. Krementulo, in reply 

The problem which deals with the minimum number of components 
used for indirect measurement of disturbances is complex and it is 
difficult to give an answer in the general case. It depends on the con¬ 
crete task and concrete control system; but usually indirect measure¬ 
ment of disturbances leads to a smaller number of components. We 
must use the methods of indirect measurement of disturbances because 
in many cases the causes of the disturbances are unknown, for example, 
m the extremum systems. 

I completely agree with the second remark. Of course, we have 
many methods of improving performance of sampled-data systems, 
but many of these methods have not been completely studied. Ex¬ 
amples of such methods are: varying the sampling rate, changing the 
s npe of pulses, etc. In this connection, I mention the interesting paper 
which was presented by Dr. B. Friedland at the 1st I.F.A.C. Congress. 
This paper deals particularly with an amplifier which has a periodically 
varying gain to improve a transient performance of the sampled-data 
servo system. 
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Optimization and Invariance in Control Systems 
with Constant and Variable Structure 
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Summary 

The paper considers invariancy in the automatic control systems 
subject to external disturbances with known statistics. Invariancy 
conditions established by means of K{D ) transform theory are gen¬ 
eralized for systems with known disturbance statistics. 

A new design principle for the systems invariant to any con¬ 
tinuous control function isdeveloped.Usually error signal is independent 
of control action only if the right-hand side of the system’s non-homo- 
genious differential equation vanishes. To meet this requirement one 
must use control action derivatives in the control algorithm. In this 
case any variation of system parameters would make these invariancy 
conditions invalid. 

It is shown that with the aid of open-loop variable structure 
systems one can perform a wide range of control functions without 
static errors. In this case there is no need to meet this classical in¬ 
variancy condition. Note that combined servo systems with variable 
open-loop structure are insensitive to certain variations of the 
system parameters. 

Sommaire 

On traite de l’invariance des systemes de commande automatique assu- 
jettis a des perturbations exterieures a caractere statistique connu. Les 
conditions d’invariance etablies a l’aide de la theorie de la transformee 
K(D) sont generalises a ces systemes. 

On developpe un nouveau principe de calcul de 1’invariance des 
systemes a l’6gard de toute fonction de commande continue. Generale- 
ment, le signal d’erreur n’est independant de Taction de commande que 
si la partie droite de liquation diflferentielle non-homogene du 
systeme disparait. Pour satisfaire a cette exigence, on doit utiliser dans 
1’algorithme de commande les derivees de Taction de commande. 
Dans ce cas, toute variation des parametres du systeme invaliderait 
ces conditions d’invariance. 

On montre qu’avec des systemes en chaine ouverte et a structure 
variable, on peut obtenir une large gamme de fonctions de commande 
sans erreurs statiques. On evite ainsi de satisfaire a la condition 
d’invariance classique. Notons d’ailleurs que ce genre de systemes 
asservis combines avec des chaines ouvertes a structure variable sont 
insensibles a certaines variations des parametres du systeme. 

Zusammenfassung 

Dieser Beitrag betrachtet die Invarianz von Regelungen, die auBeren 
Storeinflussen mit bekannter statistischer Verteilung unterliegen. 
Invarianzbedingungen, die auf Grund der Theorie der K(D) Trans¬ 
formation aufgestellt wurden, werden fur Systeme mit bekannter 
statistischer Storung verallgemeinert. 

Ein neues Bauprinzip fur Systeme, die fur beliebige, stetige Regel- 
funktionen invariant sind, wird entwickelt. Die Regelabweichung ist 
gewohnlich nur dann von der Regelwirkung unabhangig, wenn die 
rechte Seite der das System beschreibenden nicht-homogenen Dif- 
ferentialgleichung verschwindet. Zu diesem Zweck braucht man 
Ableitungen der Regelwirkung im gegebenen Regelungsalgorithmus. 
In diesem Falle wurde eine beliebige Anderung der Systemparameter 
diese Invarianzbedingungen ungultig werden lassen. 

Es wird gezeigt, daB mit Hilfe von veranderlichen Steuerungs- 
strukturen viele Regelfunktionen ohne statischen Fehler beherrscht 


werden konnen. In diesem Falle ist es nicht notwendig, die klassische 
Invarianzbedingung zu erfullen. Dabei ist zu beachten, daB kom- 
binierte Servosysteme mit veranderlichen Steuerungsstrukturen auf 
gewisse Anderungen der Systemparameter nicht ansprechen. 


Invariance and Optimization in Automatic Control Systems 

Optimization of Automatic Control Systems and K(D) Image 
Theory 

The object of the general theory of optimization of automatic 
control systems with respect to accuracy is the optimal synthesis 
of control systems operating under conditions of continuously- 
acting disturbances. 

In the deterministic set-up of the problem 1 " 3 ’ 7 ’ 8 the optimal¬ 
ity criterion is the achievement of the highest degree of accuracy 
of the automatic control system, as measured by the error e, 
which is equal to the difference between the desired g (t) and 
the realized x (t) value of the state of the system e = g (t) — x (t). 
In the case of statistical synthesis the optimal system found from 
the probability characteristics of the controlling signal and the 
interference, has a transfer function <D opt , and possesses the 
greatest accuracy only in the mean. 

The main results relating to the construction of optimal 
systems in the case of the deterministic set-up, have been 
obtained by the theory of invariance, on the basis of which 
there can be effected the construction of automatic control 
systems with an error e, equal to zero or extremely small in the 
presence of disturbances, the measurement or use of which for 
the purposes of control is feasible. The conditions of the theory 
of invariance of automatic control systems, in the case when 
disturbance links do not nullify the numerator of the transfer 
function (and thus the corresponding transfer function), and 
when / (t) is specified, are expressed with the aid of the K (D) 
image introduced by Kulebakin 

K(D)-f(t) = 0, K(D)$0, /( 0 * 0 ... ( 1 ) 

K(D) and f(t) are linked by the conditions of the operator 
K (D) image of the functions 1 . In this case for a stable system 
its transfer function must either be the conform K(D) image 
or have this operator K ( D ) image as co-multiplier. 

In the statistical set-up, with regard to the determination of the 
transfer function of a control system in the case when it has an 
infinite memory, according to the mean-square error minimum 
criterion, one of the main results was obtained by Wiener. Ob¬ 
viously, in one case it is possible to establish precisely the corre¬ 
spondence of optimal systems in the case of the statistical and 
deterministic set-up of the problem. When the dispersion / (f) 
tends to zero, Wiener’s optimal system and the optimal system 
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as determined by the conditions of invariance coincide and 
should, strictly speaking, lead to the same results. The generality 
of systems obtained in this case according to Wiener, and of 
invariant systems, in particular systems meeting the condition 
of Kulebakin’s K(D) image, are demonstrated. Taking the 
interval of observation of f(t) to be infinite, and thus being 
concerned only with the forced output of the system, the 
transfer function of a Wiener optimal system is characterized 
by the magnitude of the MS error £ 2 (ref. 6): 

« 2 =2^]^ {S„ (co) -/<D opt O)/ 2 S f (co)}dco... (2) 

S f (co) is the spectral density of f(t), S n (co) the spectral density 
of the desired output signal. In the reviewed problems of 
control for stabilization S n (co) is conformally equal to zero, 
since, with complete filtering of external disturbance /( t ]), the 
desired output of the system must be conformally equal to zero. 
The conditions of zero error e 2 min = 0 lead to the following 
requirement in respect of the optimal transfer function of an 
automatic control system: 


e 2 = 0 S n (w) = 0 (3) 

l®opt(;co)l 2 s / (co)=0 (4) 

The latter can be satisfied for <D (p) • f(t ) = 0, which is a 
sufficient condition. 

In the case indicated, when 


®Cp)= 


Mp) 
a (P) 


-0 


where (p) is the numerator of the transfer function, and A (p) 
is the characteristic polynomial of the automatic control system, 
expression (4) can be found for (a) A (p) = 0 or (b) K (p) oo, 
where K (p) is the transfer coefficient of the automatic control 
system. 

The above-mentioned conditions correspond to the known 
conditions of invariance, the realization of which in physical 
systems is determined specially. 

Without individually examining the above-mentioned 
possibilities (for O (p) = 0), the case of the non-zero operator 
<D (p) 7^ 0 will be considered. 

If ^opt ^ 0 and Sf ^ 0 the satisfaction of condition (4) 
is possible when 

®o P t(p)-/(0=0 (5) 


This requirement corresponds to the condition of invariance 
optimal according to Wiener in respect of disturbance /(/), 
and coincides with the K (D) image 1 . An analogous method is 
used to establish the community of invariant systems and 
systems optimal according to Wiener, in the case of other 
control problems. Thus the K (D) image can serve as a tool 
for automatic control systems optimization theory. 

As an example, consideration is given to the forced motion 
of an automatic control system under the influence of an external 
disturbance, which is described by the equation 


A (p)-x (t ) = (p 2 + co|) sin co K t 

The transfer function of system <D (p) = p 2 + cor 2 /A (p\ 
by virtue of condition (5) corresponds to an optimal system, 
since it contains the K (D) image of the action / (t) as a comul¬ 
tiplier (p 2 + col is the K(D) image of f(t) = sin co k t). 


Then, according to condition (4), the function |<I> (Jco )| 2 and 
S f (co) will respectively have the form of Figure 1. 

The product of the function |® (yco)| 2 S f (co) equals zero, 
since |<E> ( jco)\ 2 > 0 when co ^ cor, |0 (jco )| 2 = 0 2 when co — cor 


S f (co) = S\co — co K \ 


0 

^funct 


CO^COfc 
CO = CO K 




Figure 1 

Generalization of K (D) Image Theory for the Case of Statistically 
Given Disturbances / (t) 

The K (D) image theory expounded in the works of Kuleba- 
kin, was developed for the case of a disturbance /(r), preset as 
a determined function of time t , particularly for the class of 
functions which permit approximation of / (t), as accurate as 
desirable, by integrals of linear differential equations, homo¬ 
geneous and having constant coefficients. Shannon 9 has shown 
that a very broad class of functions, with the exception of 
hyper-transcendental functions and £ functions, may also be 
approximated by the solutions of homogeneous differential 
equations with constant coefficients. 

The need to develop statistical methods in the theory of 
invariance and in particular in the case of K(D) images is 
explained by the following. The theory of invariance with an 
accuracy up to e depends essentially upon the form of/(A The 
absolute invariance of automatic control systems in the case 
when the transfer function, as a function of/(f) equals zero, 
is generally speaking real for any / (t), constrained with respect 
to the modulus, in particular in relation to those on which in¬ 
formation is missing. 

In the case of the K (D) image the effect of absolute invariance 
may only be observed for a completely defined function /(/), 
knowledge of which, as a determined function of t , must be 
available with a probability of 1. Thus essential for the theory 
of invariance is the knowledge of /( t ), which is nesessary in 
different cases with a probability from 0 to 1, particularly when 
investigating invariance with an accuracy up to e. In the case when 
f(t) is given in a probabilistic sense, the effect of invariance— 
particularly from the viewpoint of the K (D) image theory—was 
not examined, and the theory of invariance itself is not developed 
at the present time. An attempt is made below to apply the 
theory of statistical optimization to the determination of the 
statistical probabilistic conditions of automatic control systems. 
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invariance, and generalize the theory of K(D) images to this case. 
Henceforward, as previously, we are examining the effect of in¬ 
variance, the class of statistical actions f it) and control systems 
relating only to stationary systems and stationary actions / (/)• 

Approximate Conditions of Optimalization Using the K (D) Image 
in the Case when Dispersion is Present 

In the well-known works of Kolmogorov 10 and others it is 
shown that any stationary random process may be represented 
as the limit of a sequence of processes with a discrete spectrum. 
The general expression of a stationary random process /(/) 
in this case may be as follows: 

n 

f (0~ £ a K sin(co K t + (p K ) (6) 

K= 1 

where a l9 a 2 , a z , .. ag, •. a n are uncorrelated random magni¬ 
tudes with mean value zero, i. e., 

M a = 0 i = l,2, ...,w 

M a M a = 0 i^j 

where M is the sign of the mathematical expectation. 

It is also known 6 ’ 10 that for each stationary process fit) it 
is possible to indicate a number e as small as desired and an 
observation time range T thereof, as large convenient, for as 
which there exist such pairwise uncorrelated random magnitudes 
a x ,a^...,a n that the completeness of approximation to the 

n 

series S or sin icogt + (pg), determined by the mean-square 

K= l 

difference, will be such that 

n 

£ a K sin{(o K t + (p K )\ 2 <s 

K= 1 

It has thus been shown that each stationary random process 
fit) can be approximated as accurately as desired by the sum 
of harmonic oscillations with random uncorrelated amplitude 
and phase. Most essential henceforward is the fact that co 7c 
characterizes the constant frequencies of the process/(0- 
For the above series the correlation function R f ir) has, 
as it is known, the form 

R f (t) = £ ^rCOS (0 Kt ; (M {/(0} = 0) 

K—l Z 

where a) x is the lower frequency of the spectrum of the random 
process, equal to co 1 — 2n/r max , r max is an interval of time, 
beginning with which | R f ir) | < f | R f (0) | where f is usually 
taken to equal 0*05. 

For the R f (t) under consideration, the spectral density 
S f (a>) represents a discontinuous function, consisting of 
S functions of the form 

S f (co)= £ ^<5(©-ico*|) (7) 

K=1 L 

By virtue of the foregoing, the condition of an optimal control 
system is given by the expression 

l<L P tO'ffl)l 2 £(®)=0 

or on the basis of (7) 

l^optO)! 2 - £ ^<5(<n-|<%l)=o 

X = 1 ^ 


Since the second co-multiplier of (7) characterizes the 
spectral density of some periodic function, the expression 
obtained may be written in the form 

1>opt (p)-a 1 sin(co 1 t+(p 1 )+ ® opt (p)-a 2 sin (co 2 t + cp 2 )+ ... 

+ 3> 0 p t (P) a n sin ( co n t + <?>„) =0 

In this expression the magnitudes a x , a 2 ,...,a n a.nd(p 1 ,(p 2 ,(p n 
are random, undetermined uncorrelated magnitudes, cog are 
constants for the given /( t ). For determination of <P 0 pt the fact 
that ag,(pg are unknown is not essential, since d> 0 pt ip), being 

n 

the K(D) image of fit) = 2 a K sin icog t + cp K ) is only deter- 

1 

mined by the frequency parameter co k . Since <5 0 pt if) f° r e ach 
partial frequency cog of the spectrum equals p % + cog 2 , the 
following will be the general expression of <F opt ip) 

<E>o P t (p )={ K n (p 2 + ®i)| %(p) 

where €> 0 ( p ) is the remaining comultiplier of the function <E> opt (p) 

n 

after the removal from it of II ip 2 + cog 2 ). 

K= 1 

The general problem of the approximate optimization 
®o P t ip) of a system in the presence of a random stationary 
disturbance / (/) is thus solved with the assistance of the K (D) 
image. Expansion of the stationary random process / (0 into 
series (6) is a complex problem and it should be carried out on 
the basis of a preliminary examination of the process fit). 
So henceforward consideration is given to an assumed case 
in which the process / it) can be characterized by the presence 
of several main periodic oscillations in the spectrum. In this 
case the construction of systems satisfying the condition of 
the KiD) image is facilitated by the limitation of n. In a 
number of practical examples of the use of the KiD) image 
for dynamic systems of the damping type, the conditions of the 
K iD) image are approximately satisfied only for one n = 1. 
The conditions of search of systems satisfying the requirements 
of K iD) images may be effected on the basis of the statistical 
properties of f it). In the above case the automatic control 
system under consideration must satisfy the condition 

n 

K(D )• £ a K sin (co K t + cp K )=0 (8) 

K=1 

Noting that the K(D ) image is itself invariant to random 
magnitudes of the series / (0 with random amplitude ok and 
phase cpK, and depends only on the determined values of w 7 ,, 
we shall find the K (Z>) image for (pK = -V2 and aj{ = a 1 /2 
(t — r). 

Condition (8) will then have the form 

K(D) £ * cos (o %%—0 

x — 1 z 

or K (D) R (t) = 0 where R (r) is the correlation function 
of f(t). Thus the condition of the invariance of the system 
to the disturbance fit), obtained on the basis of the theory 
of the KiD) image, is equivalent to its invariance to the 
correlation function R if) of disturbance /(f). The conclusion 
obtained is based on the expression of the stationary random 
process fit) (with a definite degree of accuracy) by a discrete 
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Kolmogorov series 6 , for which the corresponding spectral 
density is also the sum of discrete values in the form of <5 
functions. The possibility of using the discrete series (6) 
determines the applicability of the formula obtained for the 
case of an /(/) given by continuous graphs of spectral 
density. 

The condition of invariance to a random function, analogous 
to the condition derived above, can be obtained if the random 
function is not expanded into a Kolmogorov series, as was done 
above, but into a canonical series 5 *. 

The random function/(t) can be represented by its canonical 
expansion 

f (t) = m f (t)+YV B f°(t) 

V 

where m f (t) is the mathematical expectation of f(t\ which will 
henceforward be put equal to zero, V v are uncorrelated centred 
random magnitudes, coefficients of the canonical expansion, 
and f ) (t) the coordinate functions of the canonical expansion. 

The random coefficients V v in the general form of canonical 
expansion of a random function are determined by the formula 5 

V 0 = Q D F° (t) 

where i2( y ) are arbitrary linear functionals, which must satisfy 
the conditions of biorthogonality for the mutual 4 non-correlated- 
ness’ of the magnitude V v ;f° ( t ) is a centred random function. 

The condition of invariance of a control system to disturb¬ 
ance f(t) will be written in the form: 

®opt(p)'F°(t)=0 (4v(,p)^0) (9) 


Combined Tracking Systems with Variable Structure 

Combined tracking systems are one of the most significant 
spheres of application of the principle of invariance in automatic 
control. In the combined system [Figure 2 ( a )], reproduction of 
the control action g(t) is implemented with the aid of a two- 
channel system or a system with two cycles: an open-loop 
cycle , 1*2 (p) = K% (p) g (p) and a closed-loop cycle 


x(n s K, (p)K 2 (p) 
{P) 1 +K l (p)K 2 (p) 


s(p ) 


where K 2 (p) are the transfer functions of the elements of 

the closed-loop cycle, K 3 (p) the transfer function of the open- 
loop cycle, the output coordinate of the open-loop cycle and 
x the controlled coordinate. 



(a) 


The coordinate functions/^ (t) in the general form of canonical 
expansion of a random function are determined from the formula 

where D v is the dispersion of an elementary random function, 
and is an arbitrary linear functional, the lower index of 
which signifies that this functional is applied to R f ( t , r), viewed 
as a function r at a fixed value of t. 

Substituting into (9) the values of the coordinate functions 
and of coefficients V v 

®opt ( P ) • £ QJ c ° v (t) ■ ~ R f (t)=0 (10) 

(12 is a functional conjugate with 12). The expression (10) is 
represented in the form 

(t)-i®a Pl (p)R f (r) = 0 ( 11 ) 

l) r-'y 

For the identical equality of (11) to zero it is necessary and 
sufficient with /° (t) # 0, O opt (p) * 0, R f (t) # 0 that <D opt (p) 
be the K (D) image of the correlation function R (r) or contain 
it as a co-multiplier. 

However, it should be noted that the representation of 
random processes by a spectral series (or canonical expansion) 
will practically always have a limited number of terms. This 
constraint causes the appearance of non-zero deflections on 
the output of the ‘invariant’ system (non-absolute invariance). 
The evaluation of this relation has its own significance and 
is not examined here. 

* The idea of this solution belongs to A. S. Shatalov 



(b) 

Figure 2 


The transient processes in such systems can be described by the 
linear non-homogeneous differential equation M(p)e~N(p)g(t) 
where M(p) and N(p) are operator polynomials relative to p , 
p == d/d t, e is the error signal. The independence of the error 
signal from the control action g(t) is usually determined by the 
condition 

N(p) = 0 (12) 

in this case the forced component s (t) of the general solution 

force 

of the equation of the system is conformally equal to zero. The 
links with respect to the controlling action g(t) are selected in 
such a way as to satisfy condition (12). This is usually achieved 
by making the coefficients of the polynomial N(j>) consist of the 
difference of two magnitudes, one of which is determined by 
the disturbance effect (parameters of the open-loop cycle). It is 
practically impossible to satisfy condition (12) accurately. 
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An attempt will be made to solve this problem in another way. 
A tracking system will be constructed in such a way that the 
^-dimensional phase plane of a normal system of non-homo- 
geneous differential equations, by which it is described re¬ 
lative to e, where e = (e l9 s 2f ..., s n ), with control effect 
g(t) of an arbitrary type, contains some (« — 1) dimensional 
hyperplane S, and it will be required that the motion of the 
state point in S be described by a system of homogeneous 
differential equations. Then, if the state point under any initial 
conditions and for any forms of g(t) terminates its motion in 
this (n— 1) dimensional diversity of S, the error of signal e will 
always tend to zero (a—> 0) for any g(t). In other words, the 
controlled coordinate x(t) will reproduce any continuous g(t) 
without static error, and the requirements for the operator N(p), 
determined by condition (12), will be absent. If the function g(t) 
has a discontinuity at some moments, then slight dynamic 
errors will appear at these moments. An attempt will be made 
to solve this problem, using the principles of construction of 
variable-structure automatic control systems 12 . 


Conditions of Invariance in Combined Tracking Systems with 
Variable Structure 


In the domain, G , of an n dimensional space e v ..., e n let 
the motion of a dynamic system be described by a system of 
non-homogeneous differential equations with a discontinuous 
right-hand side 

§=7(e,g(0) (13) 

Here _ 

t := (f>i j * • 9 (&15 • • •? Sm )5 f (fl ? • * fn ) 

/i = £ i+1 0 = 1,2, n — l) 

n m 

/„ = - E a i £ i+ Z <Me>0(O)gi(O 

i=l i = 1 


where 


<?i(e>0(O) = 


^ for (Z c j e j)gi(0>0t. 

1 = 1 , 2 , 

bf fox gi0)<0 


n) 


Ghb^bi*, Cj are constants, gft) is a function defined and continu¬ 
ous on the whole time interval t. Let the hyperplane S , set by the 

n 

equation 2 c 3 = 0 divide the domain G into sub-domains 

n j -1 n 

G+( 2 cjsj > 0) and G~ ( 2 < 0), in which the vector 

J=i 

function /(£, g(t)) of system (13) is constrained and for any 
constant value of time t on the approach to S from G + and 
G~ there exist its limit values /+ (e, g(t)) and /“ (e, g(t)). On 
the approach of the solution e(t) to some domain U c: S let 
the vector functions f + and f~ be directed towards the hyper- 


plane S(fx > 0, f£ < 0, where /J and are the projections 
of the vectors / + and f~ on to the normal to the hyperplane 5, 
directed from G~~ to G + ). Then, when e(t) hits U there arises 
the so-called sliding mode and the solution of system (13) does 
not depend on a i9 b i9 b^.gft). In fact in this case, as shown by 
Filippov 13 , in the domain U~ there exists a solution e(t) of 
system (13), and the vector de/d t = f° (A g(t)\ where f° = 
(fi, ...,/°), lies in the hyperplane S and is determined by the 
values of the vector functions /+ and f~. 

From the condition that /° (e, g(t)) e S there follows the 
linear relationship of the components of the vector/ 0 


E cjfh 0 (14) 

7 = 1 

where/° is theyth component of the vector/ 0 whence 



n—l 


E cjf? 

7 = 1 


(15) 


Hence the solution of system (13) for e(t)e U coincides with 
the solution of the linear system of homogeneous differential 
equations 

§=7°(e) (16) 

Here 


fi (fils • • • j fin) 

fj =s/ +1 (j=l,2.n-l),/„°= —I £ C js j+1 

L'n j= 1 

Cj are constants. 

Obviously the solution of system (16) does not depend on 
a i9 b*, gj(t). Use will be made of this property of the solution 
of the system of non-homogeneous differential equations with 
a discontinuous right-hand side for the construction of a com¬ 
bined tracking system with variable structure. 

Let the structure, selected in a definite way, of the open- 
loop cycle of a combined tracking system [Figure 2(b)] change 

n 

stepwise on some hyperplane S = 2 c 3 &j — 0 in such a way 

7=1 

that the movement of this servosystem is described by a system 
of non-homogeneous differential equations with a discontinuous 
right-hand side (13), where ij/i (e, g(t)) = F (e, ^(/))] 


®i(s»S'(0) = 


(i=l,2,...,n) 

■K* for ( Z^ CjBjj gi (0 < 0 


K{ Kf are constants, determined by the open-loop cycle para¬ 
meters. It is assumed ( a ) that the domain U exists, it includes 
the origin of the coordinates, and the solution of the system of 


t In the case 



m(.e,g(t)) = bi for ^^c 3 .£ 3 .j 
Vi (?,£(0) = b* for | j 


ft <*)-» + 0, 
ft(0-»—0 


f In the case ^ ^c 3 -6 3 -| g 4 (r) = 0 


(e> g(t)) = K t for 


= for 


(^C 3 -£ 3 -)g' s -(0-S-+ 0 

( P, C J S J )ft ( 0-^—0 

\ 7=1 / 
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differential equations (16) satisfies the given requirements on 
the quality of the process of control (control time and maxi¬ 
mum dynamic error of the system must not exceed certain pre¬ 
determined values); (b) there exists a sufficiently large domain of 
initial conditions under which the solution of the system of 
equations (13) hits the domain U; (c) in the domain U there do 
not exist trajectories serving as sectors of limit cycles with a 
partially sliding regime. 

Then the solution of the initial non-homogeneous system of 
differential equations (13) will depend on the controlling action 
£i (0 and the parameters of the closed-loop and open-loop cycles 
only up to the moment when e(r) hits the domain U, where the 
solution coincides with the solution of the linear system of 
homogeneous differential equations (16) and depends only on the 
coefficients q. Thus in this case, in the reproduction of the con¬ 
trolling actions g x (t) the magnitude £ x —> 0 on a finite interval 
of time t, and the controlled coordinate x(t) reproduces g x (t) 
without static error. The quality of the process of control in 
such systems depends loosely on the variation of the parameters 
of the open-loop and closed-loop cycles, since the solution e(t) 
depends on these parameters only until it hits the domain U. It 
must be noted that in the systems under examination, the open- 
loop cycle forgj(r) # 0 in some cases exerts an influence on 
the stability of the tracking system. In particular, as an example 
will demonstrate, even when the change of the parameters of the 
closed-loop cycle leads to the loss of the stability of the closed 
loop, then on the whole for g x (t) # 0 the open-loop cycle with 
variable structure will ensure, in some domain of initial con¬ 
ditions, the stable operation of the tracking system. The above- 
listed properties of combined tracking systems with variable 
structure advantageously distinguish them from ordinary linear 
combined tracking systems. 


An Example of a Combined Tracking System with Variable Structure 

Let the equations of the individual components of a com¬ 
bined servosystem with variable structure have the form 

Mi = ki £ij T x T 2 g x (t)+(T x + T 2 )g x (t)+g x (t)~k 2 in 3 

ji 2 = (s, g, (0) gi(0 

( +Kfor(c 1 s 1 + c 2 s 2 ) g 1 (t)< 0* 
$ i(e>gi(0) = | 

{-Kfoi (c 1 s 1 + c 2 e 2 ) g x ( t) > 0 
g 1 (t)-x(t)=s 1 

where k x , k 2 , K, 7j, T z , c x , c 2 are constants. The block 
diagram of the system is depicted in Figure 3(a) and (b). In this 
case the combined tracking system, after the elimination of the 
intermediate coordinates fz x , g 2 ,/u 3 , x is described by the follow¬ 
ing system of non-homogeneous differential equations with a 
discontinuous right-hand side: 

dg _ 


* For (c x e x + c 2 s 2 )g x (t) = 0 

( £ > S'i(O) — ~t K for (c x e x -)- c 2 e 2 ) g x (t)— > + 0 
®i(e.giff)) = — K for (qq + c 2 e z ) gl (t)-+ — 0 


Here s = (e 1 ,e 2 ),g=(g 1 , g 2 , g 3 ),f = (fuf 2 ) (17) 

f 1 =e 2 ,f 2 =-2be 2 -cofa + g 3 ( t ) + 2 bg 2 (t) 

+ (O)gi(O 

Ti + T 2 


where 


2 b=- 


T x T 2 


•At (s, g x (0) 


COq— . 

1 l i 2 

_l + g>i (e,g t (Q) 


g 2 (0= 


dgi(Q 

di 


t,t 2 

S3 (0 = 


d g 2 (0 


We shall examine the behaviour of a combined tracking system 
with variable open-loop structure which reproduces various 
controlling actions g t (t), while the parameters of the transfer 
function of the closed-loop cycle K 2 (p) can be chosen within 
wide limits. 



(a) 


9,(0 



Figure 3 


The phase-plane method is used for analysis of the system. 
Let the controlling action gj(t) = A, where A is a constant and 
the parameters of the tracking system k l9 k 2 , T l9 T 2 , K are 
selected in such a way as to satisfy the following conditions: 


K-k 2 > 1 

(18) 

b 2 >co 2 0 

(19) 

^r=-b-'J(b 2 -a>l) 2 

c 2 

(20) 


Then for g x {t) > 0 the phase plane of the system will have the 
form shown in Figure 4(a ), (b) and (c). In this case, under any 
initial conditions the state point will tend to hit the straight 
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line q e x + c 2 e 2 — 0 which serves as the boundary of disconti¬ 
nuity of the right-hand side of eqn (17) while on the boundary 
of discontinuity the vector functions f + (sheet I) and/" (sheetII) 
are always directed towards this straight line and, hence, when 
the state points hits it the solution of eqn (17) coincides with 
the solution of the linear homogeneous differential equation 

(2D 

Here e = ( £l ,8 2 )J° = (/",/ 2 °) 

f? = e 2 J 2 = % 2 

C 2 

Thus the right-hand side of the equation determines the motion 
of the system only up to the moment when the state point hits 
the boundary of discontinuity, and then the motion of the 
system can be reflected by an equation without a right-hand 
side (21) or, after the appropriate transforms, by the equation 

C 1 S 1 + C 2 S 2 = 0 (22) 

In this case, therefore, static error will be absent. We shall 
follow the variation of the static and dynamic properties of the 
system as the parameters of the transfer function of the closed- 
loop cycle K 2 (p) vary. 

Let the parameters of the closed-loop cycle vary in such a way 
that the closed loop of the system becomes unstable, e.g., con¬ 
sider that the sign is altered in front of the term 2 be 2 in eqn (17). 



In this case the system will also become unstable for any para¬ 
meters of the linear transfer function of the open-loop cycle. 
When there is an open-loop cycle with a variable structure, the 
phase plane will have the form shown in Figure 5. As before, 
the state point, under any initial conditions, will hit the straight 
line (22), on which there exists a finite length mn which includes 
the origin of the coordinates 0, where the conditions of the 
existence of a sliding mode are satisfied. The tracking system 
will thus be stable. For a particular set of initial conditions 
the process will run without overshoot, and as before there 
will be no static error. Thus the variable-structure tracking 
systems under consideration are insensitive in relation to 
variation of the system parameters. 

It is not difficult to show that for g(t) < 0 all the examined 
properties of the combined tracking system with variable structure 
will remain unchanged. We shall consider whether these prop¬ 
erties of the system are preserved when reproducing other 
forms of controlling actions, e.g., g x (t) — oct, Ae a *■* where a, oc ± 
are constants. 

In this case one will be dealing with a non-stationary phase 
plane. By examining the field of the tangents to the phase 
trajectories for various fixed moments of time t, the change 
of the directions of the vector functions ft and / can be followed 
and thus the answer given to the question of the existence of a 
section of sliding mode mn on the straight line (22) and the 
landing of the state point on this section. 

Let the control action g-yit) —at. 

We shall examine the static and dynamic properties of a 
tracking system for the first case of combination of closed-loop 


427 






B. N. PETROV, G. M. ULANOV AND S. V. EMELYANOV 


cycle parameters. For the instant t = 0 [Figure 6{a)] the 
direction of the vector functions /+ and /“ in the vicinity of the 
straight line (22) is such that the section of sliding mode mn 
on straight line (22) is everywhere absent. However, with the 
time, beginning with some t = t l9 the field of the tangents 
to the phase trajectories changes in such a way that the 
vector functions /+ and f~ in the vicinity of the straight 
line (22) are everywhere directed towards this straight line 
[Figure 6(b)]. Since, with time the inclination of the tangents 
to the phase trajectories is deformed in such a way that at 
the limit it tends towards straight lines [Figure 6(c)], the 
above-mentioned static and dynamic properties will also be 
preserved when the system is reproducing a controlling 
action of the kind under review. Let the controlling effect 
gi(t) = From the analysis of the variation of the 

fields of the tangents for various instants t, it follows that 
even when reproducing a transcendental control action, static 
error is absent. 

With the aid of an electronic simulator we shall study the 
behaviour of such a combined tracking system with a variable 
structure in the reproduction of controlling actions of the form 

gi(t) — A, <x,t + A l9 A 2 e ait where A 9 oc 9 A l9 oc l9 A 2 
are constants. 

Let the parameters of the tracking system equal 

T i = h T 2 = l, k x = 1, k 2 = l, K=2 
As follows from the oscillograms in Figure 8(a), (b) and (c) 


all the controlling actions under review are reproduced without 
static errors with a good quality of the transient processes. 

We shall change any one of the parameters of the controlled 
plant, e.g., k 2 from the value k 2 = 1 to k 2 = 10, and follow the 
change of the static and dynamic errors of a combined tracking 
system with variable structure. As can be seen from the oscillo¬ 
grams in Figure 8(d), (e) and (/) the static properties of the 
system have been preserved in this; as before, it reproduces, 
without static errors, all the types of control actions under 
consideration, while the dynamic properties have not suffered 
any qualitative changes—the time of the transient processes has 
been slightly reduced. 

Conclusions 

The paper considers the invariance of automatic control 
systems in the presence of statistically given disturbances. The 
invariance conditions, obtained on the basis of the K(D) image 
theory, have been generalized for the case of statistically given 
disturbances. For stationary systems of automatic control and 
stationary disturbances f(t) the conditions of the K(D) images 
in relation to the disturbance prove to be equivalent to the 
condition of the K(D) image in relation to its correlation func¬ 
tion. 

A new principle has been proposed for the design of in¬ 
variant tracking systems in relation to continuous functions of the 
controlling action, which ensure the absence of static error. It is 
shown when using an open-loop cycle with variable structure 
that it is possible to reproduce, without static errors, an extensive 
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Figure 8 



class of controlling-action functions. When selecting the open- 
loop cycle transfer function there is no need to satisfy the 
classical conditions of invariance, which require the right-hand 
side of the non-homogeneous differential equation to vanish. 
This property of the systems under consideration makes it 
possible to build invariant tracking systems without differentia¬ 
tion of controlling action. The variable-structure combined 
tracking systems considered are insensitive to the variation of 
the system parameters within a certain range. 
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DISCUSSION 


J. E. Gibson, 168, Drury Lane, West Lafayette, Indiana, U.S.A. 

It appears that it would be difficult or impossible to design a system 
which satisfies the law of eqn (12) of the paper. 

S, Emelyanov, in reply 

It is, in fact, impossible that the condition (12) be strictly satisfied in 
the real linear systems, because even very small variations of para¬ 
meters will lead to its deterioration. 

The main point of the paper is a development of such systems which 
would in some way have properties of linear systems under a strict 
satisfaction of condition (12) (astatic realization of any analogue 


function of control action). At the same time, they should not have 
the shortcomings of linear systems [difficulties of strict satisfaction of 
condition (12)]. This problem can be solved by the application of 
control systems with a variable structure. They can provide the condi¬ 
tions under which the phase space of the system developed will include 
some (n — 1) dimensional subspace, where the motion would be ex¬ 
pressed by homogeneous differential equations. 

In this case the control actions will be realized without statistical 
errors. Besides, the variations of system parameters in a wide range 
will not give a stable error and will not deteriorate the transfer function 
of the process as it follows from the example given in the paper. 
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Synthesis of Systems with Fixed Structures of 
Equivalent Self-adjusting Systems 

M.V. MEEROV 


Summary 


The paper describes the methods of designing automatic control 
systems with constant structures and constant parameters, the prop¬ 
erties of which are equivalent to one of the adaptive systems. 

These systems are based on the structures which admit an un¬ 
limited increase of gains without disturbing the stability. 

It is shown that the presence of the disturbance at the input of 
the compensating network (when the disturbance can be measured) 
or the introduction of the model of the plant to the system provides, 
for the chosen structure of the whole system, the removal of the 
influence of external disturbances. 

It is concluded that the sensitivity, according to Bode, for the 
mentioned structures does not depend on the parameters of the plant, 
nor does its value tend to zero under the corresponding increase of the 
gain. 

Sommaire 

On decrit les methodes de calcul de systemes de commande auto- 
matique a structures et a parametres constants, dont les proprietes 
sont equivalentes a delies de systemes adaptatifs. Dans ces systemes, 
une augmentation illimitee de gain ne perturbe pas la stabilite. 

En cas de perturbations mesurables a l’entree du reseau compensa- 
teur, ou de finiroduction du modele de Installation commandee leur 
influence se trouve eliminee avec une structure d6finie d’avance 
pour le systeme global. 

Dans ces systemes, la sensibilite. au sens de Bode ne depend pas 
des parametres de installation, et sa valeur ne tend pas vers zero 
quand le gain augmente d’une maniere correspondante. 

Zusammenfassung 

Der Aufsatz beschreibt Entwurfsmethoden fur automatische Regel- 
systeme mit fester Struktur und konstanten Parametern, deren Eigen- 
schaften einem der selbsteinstellenden Systeme entsprechen. 

Derartige Systeme beruhen auf Strukturen, die eine unbegrenzte 
Zunahme der Verstarkung zulassen, ohne instabil zu werden. 

Wie die Arbeit zeigt, lassen sich die auBeren Storeinflusse einmal 
ausschalten, indem man sie als zusatzliches Eingangssignal (wenn die 
Stoning meBbar ist) dem kompensierenden Netzwerk zufiihrt; zum 
andern gelingt die Unterdriickung der Storeinflusse durch Hinzu- 
ftigung eines Modelles der Strecke zum gewahlten Gesamtsystem. 

Es wird daraus geschlossen, daB fur die erwahnten Strukturen die 
Empfindlichkeit nach Bode weder von den Parametern der Strecke 
abhangt noch deren Wert gegen Null strebt, wenn eine entsprechende 
Zunahme der Verstarkung vorliegt. 


The problem of ‘self-adjustment’ in a control system arises in 
connection with the fact that in the operating process the 
characteristics of the controlled plant may vary within a wide 
range. Under these conditions, the adjustment of the control 
system, or even its structure, which was entirely expedient for the 
initial form of the plant’s characteristics, may prove to be 
completely unsatisfactory for-the altered characteristics. 


A change in the characteristics of the plant to be controlled 
may be conditioned, basically, by the two most prevalent 
factors. In the first place, a change in the characteristics may be 
brought about by external disturbances that are applied to the 
plant; and in the second place, a change in the plant’s character¬ 
istics may take place in the course of its operation. 

The problem of ‘self-adjustment’ also comes up in a number 
of cases where the information regarding the characteristics and 
properties of the plant is insufficient; it is only known that the 
plant’s characteristics have an extremum for some quality 
criterion, and the control system’s problem consists in a search 
for this extremum and in maintaining the plant’s operating 
conditions at this extremum. A rather large number of studies 
(for example, by Feldbaum 1 ’ 2 > 4 , and by Doganovskii and 
Feldbaum 3 ) have been devoted to methods of searching for, 
and adjustment of the system to, the disclosed extremum. In the 
present paper, methods are considered for the purpose of 
constructing systems with fixed structures that would 
maintain the most favourable adjustment, independently of 
external disturbances, and the character of which may be 
practically arbitrary. The sole limitation is the one regarding 
disturbances in accordance with the modulus (absolute value). 
In the present study, no consideration is given to the method 
of searching for the extremal condition for some quality 
criterion. If, however, the extremum is established by some 
method or other, then the structures examined below maintain 
this extremum automatically, without the need for a duplicate 
search. 

Methods of Constructing Control Systems for the Case where the 
External Disturbances may be Measured 

Consider the automatic control system whose block dia¬ 
gram is shown in Figure 1. In this diagram the designation 
w 2 ( p ) is given to the transfer function of the controlled plant, 
kwi(p) and w z (p) to the transfer functions of the control 
system and of the stabilizing device, and F to the external 
disturbance. kw 1 and h> 3 ( p ) have been selected in such a way 



Figure 1 
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that, in the absence of disturbances i 7 , the process, which is 
the most favourable from the point of view of the selected 
quality criterion, is attained in the case of a sufficiently large 
gain, k. Thus, for example, the optimum operating con¬ 
ditions are realized where there is an unlimited increase in 
the gain, the plant is non-linear, and there is a non-linear 
feedback in the optimum control circuit with an automatic 
potentiometer 5 ^ 6 . It is natural for the designed circuit to remain 
stable when there is an unlimited increase in the gain. The 
following proof is given: the structure, which is shown in 
Figure 1 , giving no consideration to external disturbances 
and where k tends to infinity, is equivalent to the system in 
Figure 2, where consideration is given to disturbances and 
where k tends to infinity. In other words, in order to elim¬ 
inate the effect of external disturbances that are capable 
of being measured, they should be supplied to the input of 
the stabilizing device in the form of an additional signal. 


from which: 


/cw t (p) w 2 (p)x 

input (fe) + w 2 (p)F(p) 
1 + kw t (p) w 2 (p) + few, (p) w 3 (p) 


( 6 ) 


When k tends to infinity, one obtains: 

/ x W i(P)W2(P)X ia p ul (p) 

output w, (p) w 2 (p)+ w, (p) w 3 (p) 
or 

(7 ) 

*-00 Xinput(p) W 2 (p) + W 3 (p ) 

i.e. exactly the same expression as eqn (2). From what has been 
obtained it follows that the system in Figure 2 , when there is a 
sufficiently large gain, will behave as a self-adjusting one, in 
the sense that its characteristics will remain unchanged despite 
the presence of external disturbances whose character is practi¬ 
cally unlimited. 



Figure 2 


Actually, the transfer function of the system in Figure 1 , 
without calculating the external disturbances, will have the 
form: 


^input 


GO 

[p] 

lcw t (p ) 


Xinnu ,(p) 


I+ZcwjO) w 3 (p) 


W 2 (p) 


kw 1 (p)w 2 (p) 


1 +- 


(p) w 2 (p) l + fcw 1 (p)w 3 (p) + /cw 1 (p)w 2 (p) 

l + /cw 1 (p)w 3 (p) (1) 

Assume that k tends to infinity; then, 

w 1 (p)w 2 (p) w 2 (p) 


v ad justed 


(p)= 


Wi (p) W 3 (p) + w, (p) w 2 (p) w 3 (p) + w 2 (p) 


( 2 ) 


Now, the transfer function for the circuit in Figure 2 is found; 
one has: 


Y(p) = kw l (p){x ; 


input -^output [Y(p)+F(p)]w 3 (p)} 




output 


(p) = w 2 (p)[F(p)+F(p)] 


(3) 

(4) 


(5) 


From (3), 

Y( P ) = 

input (p) — kw 1 (p) 

•^output (p)-few 1 (p)w 3 (p)F(p) 

1 + few,(p) w 3 (p) 

Substituting the value Y ( p ) from (5) into (4), one obtains: 

[l + fcW!(p)w 3 (p)] 

•^output (p) 

= kw 2 (p) w 2 (p) x input (p) - kw t (p) w 2 (p) x output (p) F(p ) 
- few, (p) w 2 (p) w 3 (p) F(p) + w 2 (p) F(p) + few,(p) w 2 (p) w 3 (p) 


Methods of Constructing Structures for the Case where it Does 
Not Appear Possible to Measure Disturbances Directly 

Now consider the case where the plant’s characteristics 
change due to the effect of external disturbances, but where it 
does not appear possible to measure these disturbances. Such 
a situation is, for all practical cases, highly prevalent. 
A series of disturbances is difficult to measure, in the first place, 
because of the properties of the disturbances themselves, and 
in the second place because of the absence of sufficiently high¬ 
speed measuring devices for the measurement of the external 
disturbances. 

The solution of the problem in the given case is carried out 
in the following fashion. Assume that the plant’s character¬ 
istics are known for the case where disturbances are absent. 
For this case, a control system is constructed in such a way that 
the optimum operating conditions should be attained when 
there is an unlimited increase in the gain, k. Strictly speaking, 
in the absence of disturbances, the system has the form shown 
by Figure 1 . As was indicated earlier in this paper, in the case 
where 1c tends to infinity, one has: 

k&dj. ( P ) = [>2 0)]/[w 3 (p) + w 2 (p)] 

Now Figure 3 is plotted. The output of the controlling part 
of the circuit, which is designated in Figure 3 by the letter Y, is 
fed to the input of the real plant and to the input of the model 
with a transfer function w' 2 (p ). In future, w' 2 ( p) is called the 
transfer function of an ideal plant. 



Figure 3 w convert. (P) 
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The difference between the outputs of the ideal and real 
plants is fed through a converting device with a transfer 
function w convert O), to the input of the stabilizing device. Now 
the transfer function of the system in Figure 3 is found. 

Y(p) = kw t (p) (x !nput (p) - x output (p) 

- w 3 (p) [ y (p) + (x output (p) - 

•^output 00) ^"convert m (8) 

^output (p) = w 2 (p) [Y(p)+F (p)] (9) 

^output (p) = w 2 (p)Y(p) (10) 

On the basis of (9) and (10), one may write: 

•^Output iP) ^output ip) 

= w 2 (p)[Y(p)+F(p)-Y (p)] = w 2 (p) F (p) (11) 

by calculating (11), eqn (8) is written as: 

Y(p) = kw 1 (p) {x input (p) - x outpm (p) 

-w 3 (p)y(p)-w 3 (p) 

^convert (T)w 2 (p)T(p)} (12) 

From (12), the expression for Y(p) is found, namely: 

Eqn (13) * 

By substituting the expression for Y(p) from (13) in (9), one 
obtains, after some elementary calculations: 


Generally speaking, the elimination of the influence of 
disturbances, in the given case, could be accomplished by the 
method described by the author 7 . Naturally it is expedient to 
make use of the indicated method if there are no additional 
disturbances at the system’s input. If, at the system’s input, 
there are disturbances in addition to the useful signal, then it is 
possible to show that the solution given here is more noise-proof. 
Let us convince ourselves of the accuracy of this affirmation. 

It is assumed that, in place of the transfer function kw ± (p\ 
and in place of a stabilizing device with a transfer function 
w 3 ( P X which provides for stability in the presence of an un¬ 
limited gain k , a system having the form shown in Figure 4 
is realized. 

The introduction of an amplifier, with a high gain, which 
is encompassed by a stabilizing device with a transfer func¬ 
tion w 3 , depends on the necessity of providing stability to 
the entire system in the presence of unlimited increase in k. 



[1 + kw l iP) W 3 ip) + fcWi (p) W 2 if)] X output (p) 

= k w 1 (p)w 2 (p)x input (p) 

-kw 1 (p)wi(p) 

^convert (p) W 3 (P)F(p) 

+ w 2 ip) F (p) + kw 1 (p) w 2 (p) w 3 (p) F (p)... (14) 

Assume that the transfer function of the stabilizing device 
has been selected in such a way that the structure obtained 
assures stability where there exists an unlimited increase in the 
gain, k. Dividing eqn (14) by kw x (p), and assuming that kw^p) 
tends to infinity, one obtains, after some simplifications: 

[w 3 (p) + w 2 (p)] 

•^output (p) = w 2 (p)x input (p) 

+ L W 2 iP) w 3 iP) ~ W 2 (p) 

^convert (p)w 1 (p)]F(p)... (15) 

As is evident from (15), in order to eliminate the effect of disturb¬ 
ances, the transfer function of the converting device should 
be selected from the condition: 

w 2 ip) w 3 (p) - w 2 (p) w 00nvert (p) w 3 (p)=0 (16) 

or: 

^convert ( P ) ~^/^2 ( P ) 

The realization of a device with a transfer function (16) may be 
attained by methods of constructing structures that are stable in 
the presence of an unlimited increase in the gain 6 , and this in¬ 
volves neither fundamental nor technical difficulties. 


Figure 4 


If w 2 (p) has a power in the denominator that is greater than 
a ‘fourth’ one, then, as is shown 6 , it is possible to introduce 
several amplifiers with high gains and realize a structure that 
would admit an unlimited increase in k without disturbing the 
stability. 

As is clear from Ref. 7 in this case, if x input contains no dis¬ 
turbances, then an increase in the gain k, up to rather high 
values, eliminates the effect of the disturbances F. 

Assume that the input signal contains a disturbance /input 
to the extent that 

•^input ^input u "h/input 

where .xinput u is the disturbance-free input signal. 

A system of equations for the circuit in Figure 4 is drawn up, 
for the case under examination. At the same time, instead of the 
part of the circuit surrounded by a dotted line in Figure 4 , 
assuming that here k is a sufficiently large number, one should 
straightway insert l/w 3 (/?). 


^1 iP) - k [*inp U t u iP) +/input iP) ~ ^output ip)] ■ ■ 

1 


y 2 (p)=y 1 (p) 


™*ip) 


k 


*>3 iP) 


(17) 


E^inputu (t) ~h f input O) •^output ip)] (18) 


^output iP ) ^2 ip ) C 1*2 ip) F (p)] (19) 


* Eqn (13): 


Y (p) = —1 M ip) ~ kw ! ip) ^output ip) - kWl (p) w 2 (p) w convert (p) w 3 (p) F (p) 

i + /cw 1 (p)w 3 (p) - — 


( 13 ) 
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Substituting the value Y 2 (p) from (18) in eqn (19), one obtains: 

V (p) x inputu(p) , ( input (P) 

A output \P) ~~ —— -1- {p) -— 

W 3 (p) w' 3 (p) 


or: 


1 + 


+ w 2 (p)F(p) 
kw 2 (p) 


kx 


'output 


(p) 


W3O0 


w 2 (p) 


w, (p) J 

kw 2 (p) 


output (p) 

D^input w (P) +/input GO] + w 2 GO F GO 


W 3 (P) 

When k tends to infinity, one obtains: 

^GO _^(p)fv (rAj.f ( w 

W 3 (p) ° UtPU ‘ w ^ (p) i - X inpm U (P)+/i„pu t (P)J 

0r " X output GO = ^input u GO +/input GO (20) 


Consequently, one obtains at the output a magnitude that 
is equal to the ideal input plus the disturbance. 

Consider, at this point, the size of the magnitude at the 
output, in the presence of interference at the system’s input, and 
with the elimination of the effect of disturbance F by the above- 
mentioned method. 

Keeping in mind that at the input of the system in Figure 4 , 
along with the useful signal, there is a disturbance input, one 
has the following system of equations (Figure 3) 

[ * 


Eqn (21) 


or, considering ( 11 ), one has: 

from which: 


Eqn (22) 


Eqn (23) 


Substituting the value of y (p) from (23) in (9), and after some 
elementary calculations, one obtains: 

l§ 


Eqn (24) 


On fulfilling the condition w 2 (p) = l/w 8 ( p ) and when k tends 
to infinity, one obtains: 

w 2 (p) 


A output 


(P) = 


r'X 


+ 


■w 3 (p) + w 2 (p) " input 
(p) 

W 3 (p) + w 2 (p) 


u(P) 

if input (j?) 


(25) 


By comparing the results expressed in eqn (20) and in eqn (25), 
one can draw the following conclusions. In the first case (eqn 20), 
the greater the gain, the closer the output magnitude to the 
sum of the ideal input plus the full disturbance at the input. In 
the second case (eqn 25), the picture is different. Depending on 
the properties of the useful signal and of the disturbance, 
especially for those cases where the frequency properties of 
the disturbance and the useful signal are different, the para¬ 
meters w z (p) may be selected in such a way as to reduce the 
interference, at the input, together with the useful signal, to a 
minimum. 

A Change in the Plant’s Parameters Taking Place as a Result of 
a Change in Operating Conditions or in Internal Factors 

This case pertains to plants, in which, in the course of 
operation, the parameters of the plant itself may vary within 
a wide range. In such cases, the sensitivity factor, according 
to Bode 8 , represents an essential quality index of the entire 
system. For the plants being considered here, the sensitivity 
factor may be expressed in the following manner. 

Assume that the plant’s transfer function, as before, is 
designated by w 2 (p). The overall transfer function of the entire 
system i.i relation to changes in the plant being indicated by 
k (p), the sensitivity, is expressed in the following way: 

dk(p) 

c*(p) _ k (p) _ d/c( p)w 2 (p) 

W2(P) dw 2 (p) dw 2 (p) k(p) ( 

w 2 (p) 

In the general case, the smaller the magnitude of ( p ), the 

less sensitive are the dynamic properties of the system, in its 
entirety, to changes in the plant’s properties. For this case, the 
system is considered ideal or self-adjusting, if the magnitude 
s w%) does not depend on the characteristics of w 2 (p) or S 7 ^ P) 
tends to 0 . 

The following proof is given. Structures that are stable in 
face of an unlimited increase of gain, in which stability is 


i 



Figure 5 


* Eqn (21): y( p ) = fcvty (p) {x input „ (p) + / input (p) - x output (p) - w 3 (p) [y (p) + (x outpul (p) - x;„ tput (p)) w convert (p)]} (21) 

f Eqn (22): y ( p j = fe Wl (p) [x input „ (p) - / input (p) - x output (p) - w 3 (p) y (p) - w 3 (p) w convert (p) w 2 (p) F (p)] (22) 

+ Eqn (23): kw^ (p) [x inputu (p) + / input (p) -x oulput (p) - lew 3 (p) w 2 (p) w 3 (p) w convert (p) J 1 ] 

l + /cw 1 (p)w 3 (p) 

§ Eqn (24): 

t _ fcty l (?) w 2 (p) Xinput u (p) ± kw l (p) + /input (?) - few t Q>) w| (p) W convert (p) W 3 (p) F (p) + kw 1 (p) W 2 (p) W 3 (p) F (p) 

1 + kw[ (p) w 3 (p) + kw t (p) w 2 (p) 


(23) 


^output' 
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achieved by the introduction of ideal derivatives and whose 
degree of ideality is determined by the magnitude of the gain 7 , 
belong to the category of self-adjusting systems in the sense 
indicated above. There is no question about that. In Figure 5 
one observes the structure of a control system of the type under 
consideration. The transfer function of the closed loop will be 
written in the following form: 


k(p) = 


w 3 (p) 


w(p) 


1 -f* 


w 3 (p) 


w 2 (p) 


(27) 


Now the expression for the sensitivity is found. In conformity 
with (26): 



w- 


l Q0(i+ 


kwz (p) 
w 3 (p) 


w 3 (p) 


w 2 (p) 


or, after simplification: 


c* ( p ) _ 

^2 (p) — 


1 + 


k 


W 3 ( p ) 


w 2 (p) 


' (28) 


When k tends to infinity, S^ ] v) tends to 0. In other words, 
in the sense indicated above, one obtains an ideal system. 

Now consider the expression for sensitivity, if the structure 
belongs to the category of those that are stable in the face of an 
unlimited increase of gain, and where stability is achieved by the 
introduction of passive stabilizing devices. 

As an example, one should consider the simplest type of 
such a system whose structure is shown in Figure 6 (excluding 
the dotted line part). 


kw^p) w 2 (p) 



The transfer function of the closed loop in Figure 6 is 
written as follows: 


k(p) = 


_ fcwt (p) w 2 (p) _ 

1 + kw 1 (p) w 3 (p) + kw 1 (p) w 2 (p ) 


(29) 


The sensitivity, according to w 2 ( p), is written: 


Eqn (30) 


or, after simplification: 


Eqn (30a) 


f 


When k tends to infinity, one has: 


lim S k (p) - W 3(P) 

f-3, 00 w 2 Cp)+ W3 (p) 


(31) 


Consequently, in the given case, even with sufficiently high 
gains, a change in the parameters or characteristics of the plant 
exerts an influence on the dynamic properties of the system. 

Consider some methods for improving the system’s structure, 
with the object of reducing to the minimum the effect of the 
variation in the plant’s characteristics on the system’s dynamic 
properties, and in this manner, make the system self-adjusting 
in the above-determined sense. 

When external disturbances, which did not seem capable of 
measurement, act on the plant, it is expedient in this case, 
too, to introduce a plant model into the system, in order to 
obtain a self-adjusting system. A structural block diagram for the 
case under consideration is shown in Figure 6 (including the 
dotted line part). 

Keeping in mind the designations set forth in Figure 6, one 
writes: 

~Eqn TIT) I * 


Here, .x'outputO?) is the representation of the output of the plant’s 
model and x output is the representation of the plant’s output. 

It is assumed that the model’s characteristics remain invariable. 
Under these conditions, the difference x' outpu t (p) ~~ ^output (p) 
is equivalent to the disturbance which depends on the change in 
the plant’s characteristics. Consequently 

^output (p)~ ^output (p) = CF(p) (33) 

C is a constant coefficient. 

In this manner. 


Figure 6 


^output (p)= w 2 GO y GO=w 2 GO y (p) + Cw' 2 (p) f go (34) 


* Eqn (30): 

f Eqn (30a): 

* Eqn (32): 


S k( P ) _ kwi GO [fewi GO GO+GO ^2 GO+1] - kw t (p) kw 1 (p) w 2 (p) 

W2 (P) [l+kw t (p) w 3 (p) + kw t (p) w 2 (p)] 


C k(p) _ 
(p) 


(p) • kw 1 (p) w 3 (p) + kw t (p) 


kw t (p) w 3 (p) 


kw 1 (p) [1 + kw 2 (p) w 3 (p) + kw 1 (p) w 2 (p) 1 + kw 2 (p) w 3 (p) + kw 2 (p) w 2 (p) 


Y (p) = kw 2 (p) [x inpm (p) - x output - w 3 (p) y - w 3 (p) (x output (p) - x; ulput (p))] 


(30) 

(30 a) 
(32) 


434 











SYNTHESIS OF SYSTEMS WITH FIXED STRUCTURES OF EQUIVALENT SELF-ADJUSTING SYSTEMS 


Substituting in eqn (32), instead of xr'output (jp) - ^output ( p ), 
the difference value from (33), one obtains: 


Eqn (35) 


From the above, the expression for Y ( p ) is found: 

, _fcwi(p)x inp u t (p) - kw l (p)x output (p) - kw l (p')w 3 (p)CF(p) 


Y(P)= 


1 +kw 1 (p)w 3 (p) 


(36) 


Substituting the value for y ( p ) from (36) in eqn (34), one obtains: 

It 


Eqn (37) 


or, determining x output (p) from (37), one obtains: 


cases involving changes in the characteristics of the controlled 
plants—changes due to the effect of external disturbances, which 
could be measured, those due to external disturbances that 
do not appear to be capable of measurement, and those which 
result from plant characteristic changes in the course of oper¬ 
ation that are independent of external disturbances—methods 
are suggested for designing structures which would provide for 
the independence of the plant’s selected operating conditions 
from possible external and internal effects on it, and, con¬ 
sequently, the structures obtained prove to be self-adjusting 
system structures. 

References 


(P) - M (38) 

1 +Kj ipiwUpi-'rkw, !p) w-Jp) 

When k tends to infinity: 

x t (p) W l(P) W 2G?)* in p ut (p) W-sOQXinputO) 

w 1 (p)w 3 (p) + w 1 (p)w' 2 (p) W 3 (p) + w' 2 (p ) (39) 

From (39) it is evident that the output magnitude does not 
depend on the change in parameters of the controlling device. 
Under the conditions where W ( p ) corresponds to the optimum 
operating circumstances, from the point of view of some 
quality criterion, the process in the system will be maintained 
automatically at these working conditions, independently of the 
plant’s characteristic changes. 

Thus, as a result of considering the three most interesting 
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* Eqn (35): 
t Eqn (37): 


^output " 


Y (p) = kw ! (p) [* input (p) - x ootput (p) - w 3 (p) y (p) - w 3 {p) CF (p)] 
_ w' 2 (p) [fcwi(p) 

•^input (p)-kw 1 (p) 

-^output (A>-w 3 (p)/cwi(p) CF(p) 


l + kw 1 (p)w 3 (p) 


+ w' 2 (p)CF(p ) 


(35) 

(37) 


DISCUSSION 


D. A. Bell, A.M.F. British Research Laboratory, Reading, Berks., 

England 

While I agree with most of Professor Meerov’s conclusions, which 
have something in common with the work of Horowitz 1 ,1 have doubts 
about the effects of gross non-linearities. A moderate degree of non¬ 
linearity can be handled by the method of describing functions, but 
if there is complete saturation, I believe this invalidates the whole 
concept of (linear differential equation) transfer functions in terms of 
which the whole of the author’s work is presented. Is this point dis¬ 
cussed in Reference 6, and has that reference been translated into 
languages other than Russian ? 

I do not agree that an ideal servo system is ipso facto a ‘self- 
adjusting’ servo. I suggest first that a servo system is a closed-loop 
system, and that ‘closed loop’ implies normally the existence of uni¬ 
directional paths and a finite time of propagation of signals round the 
loop. (This distinguishes a closed-loop system from an action-reaction 
situation.) Such a closed-loop system has a transfer function, and I 
would define an adaptive servo, or one of which the transfer function 
is automatically modified, as a result of observation of its performance. 

The author refers in connection with eqn (25) to the noise-filtering 
properties of a servo, and I believe the major use for adaptive servo 


systems is where the statistical characteristics of either signal or noise 
are liable to change. 

The other limitation of the author’s ideal systems is that they 
require infinite gain-bandwidth in part, at least, of the loop, bearing 
in mind the restrictions imposed by signal level and linearity require¬ 
ments. I doubt whether virtually infinite gain-bandwidth is always 
attainable. An adaptive system may then be needed to shift the available 
gain-bandwidth of the control amplifier into the band in which it is 
currently required by a varying controlled system. 

Reference 

1 Horowitz, J. M. Plant adaptive systems versus ordinary feedback 
systems. Trans. Inst. Radio. Engrs. AC-1, No. 1. (1962) 48 

J. B. Cruz, University of Illinois , Champaign, Illinois , U.S.A. 

The structures proposed by the author are quite interesting. Although 
the derivations of the expressions for the output quantities for the 
structures proposed by the author are correct, all the results could be 
written down in one step if Mason’s 1 rule for signal flow graphs is used. 
In particular, one could avoid the intermediate eqns (3), (4) and (5), 
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to obtain eqn (6); eqns (8), (9), (10), (11), (12) and (13) leading to 
eqn (14) as well as eqns (17), (18), (19) and other (unnumbered) equa¬ 
tions after (19) could be deleted. Likewise, eqns (21), (22), (23) and 
(24) which are two columns wide, could be avoided. It appears that 
using Mason’s rule, the length of the paper could be reduced by a 
factor of 2. 

The second case considered by the author, where the disturbance 
input to the plant cannot be measured directly, and the third case, 
where there are parameter changes in the plant, are very directly 
related. The structure of Figure 3 which is the one proposed for case 2 
as well as case 3 (except that >v C onvert * ( p ) is absent) is reducible to a 
typical two-degree-of-freedom structure considered by Horowitz 2 . It 
is well known that for two-degree-of-freedom systems such as those 
used by Professor Meerov in cases 2 and 3, the problem of minimizing 
the effects of disturbance inputs is not independent of the problem of 
minimizing the effects of plant parameter variations. As a matter of fact, 
if the loop gain is made to approach infinity, the sensitivity to para¬ 
meter variations goes to zero and the effect of plant disturbance inputs 
also goes to zero 2 . However, it is tacitly assumed that the system remains 
stable. If the equivalent canonic two-degree-of-freedom structure be¬ 
comes unstable as the loop gain is increased, then the structure pro¬ 
posed by the author could be unstable since the transfer function is 
not changed. Making the loop gain as high as stability will allow is, of 
course, well known in feedback system design, if low sensitivity is a 
major objective. 

The third comment I make is with respect to the assumption that 
all controllers are realized exactly and amplifier gain can be made 
infinite. Since it is not obvious whether deviations from these assump¬ 
tions correspond to second-order effects only, I ask Professor Meerov 
if he has pursued these matters further. 
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G. Schmidt, Institut fur Regelungstechnik, Schlofigraben 7, Technische 
Hochschule Darmstadt, W. Germany 

My remark concerns the last part of the paper. All the control systems 
considered by the author may be reduced to the single loop two-degree- 
of-freedom structures investigated by Horowitz 1 and others. There 
exists a very simple connection between the sensitivity function and 
the control elements. 



Figure A 

The transfer function of the system shown in Figure A is 

k ( p ) = :?CQUt P ut 00 w 2 (p) 

•^input GO l+w 1 O)w 2 0>)w 3 (p) 

From this expression one can find by differentiation, and after some 
simplification, the sensitivity function corresponding to w 2 (p): 

dfcQQ 

s t( P ) _ k(p) _1_ 

W2(p) dw 2 (p) 1 +w 1 (p)w 2 (p)w 3 (p) 

w 2 G0 

It is evident that the denominator of this sensitivity function contains 


only the open-loop transfer function of the control system. Consequent¬ 
ly eqns (28) and (31) of the paper may be written in a much simpler 
way than was done by the author. 

Reference 

1 Horowitz, J. M. Synthesis of Feedback Systems. 1963. New York; 
Academic Press 

J. Williams, Engineering Laboratory, Oxford University, England 

Feedback systems are, by their very nature, ‘self-adaptive’ and the 
fundamental reason for using feedback in the first instance was that 
the output of such a system would be relatively insensitive to para¬ 
meter changes. To illustrate that one can obtain an invariant system 
with feedback: consider Figure A. 



f (p) 


Figure A 

Let M ( p ) represent the transfer function of a model which has 
been designed to give a desired performance under all conditions of 
operation. H(p) is the transfer function of the plant and may include 
stabilizing networks so that large values of the amplification factor K 
may be used. F(p) is the transfer function of the feedback network. 

We require: Z (p) = Y m (p) — Y(p)=0 
Substituting for Y m ( p ) and Y (p) 

Z( P )-X(p)[ M (p)- 1 - + |^ (p) ~ 

For large K: 

Z(p) = X(p)[m(p)- j L- ) 

For Z(p) — 0: 

M(p)F(p) = 1 

Therefore if the feedback network is made inverse of the model, and 
provided K is sufficiently large, the output of the system will be equal 
to the model (desired) output. 


M. V. Meerov, in reply 

To begin with, I single out the remarks of Mr. Williams who expressed 
the view that, generally, a system with feedback is, by its nature, 
adaptive. This is true, but in modern automatic control theory and 
practice the concept of self-adjustment has a somewhat different sense. 

Although, as will be seen below (and it seems to me that this fol¬ 
lows from my paper and that of Mr. Kochenburger), for a system to 
be adaptive it is not necessary that it contains search elements, it 
must nevertheless possess something more than an ordinary system 
with feedback. 

The other three comments—by J. B. Cruz, D. A. Bell and G. Schmidt 
—concern the following points: 

(1) It is assumed in my paper that the system remains stable with 
an increase in the gain. The point raised is how this is and what 
has been published on the theory of construction of systems which 
are stable with unlimited increase in gain. 

(2) The question of non-linearity. 
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(3) The connection between the results in my paper and those of 
J. M. Horowitz. 

(4) Are the systems described in my paper adaptive? 

In 1947 1 I published a paper proving the existence of a class of 
structures for which there was no contradiction between stability and 
precision. In systems with such structures, it is possible to increase the 
gain without constraint, without impairing stability. Methods were 
derived for building such structures and their principal dynamic 
properties ascertained. 

A number of other studies in this direction were subsequently 
published. My book 2 summed up all the previous studies. The ideas 
in the paper are based mainly on the results of this book, which also 
examines questions connected with the evaluation of the influence of 
certain non-linearities. Owing to lack of space I was, of course, unable 
in the paper to make more than brief references to the material in the 
book. However, this book is now being translated into English, and 
will be published shortly. 1 would add that some of the relevant mate¬ 
rial may also be read elsewhere 3 . These references should answer the 
majority of these points. 

On the third question the answer is straightforward. It is shown in 
the paper that among the structures which are stable, however great 
gain, there is a subclass for which —■/£ (sensitivity according to 
Baudet) automatically (structurally) tends to zero. 

Horowitz has shown that to make /£ small, the structure must be 
two-dimensional. The difference between my work and that of Horo¬ 
witz is therefore obvious. 

In reply to the final question, an illustration shows the practical 
significance of the structures obtained. 


No one will doubt that the system proposed by Kochenburger is 

adaptive, so 3 shall give the solution of Kochenburger’s problem by 

my method. g 

There is a plant with transfer function .. , - -.r -■ -- —. n AAC "T 

(1 -f 0*2 p) 2 (1 -f 0-005 p) 

where, in the process of operation, K can change 100 times, and, 

moreover, quite rapidly. 

K 


Ki=200 K2=200 K3 = 200 (1+0 2p) 2 (1+0 005p) 



Figure A 


Kochenburger’s solution is well known, being contained in his 
paper read at this I.F.A.C. Congress. Figure A shows the layout and 
data of the individual elements of the circuit which solves the same 
problem by our method, = T 2 = 0*01 sec. 

Figure B shows the oscillogram for K — 10, and Figure C the 
oscillogram for K — 0*1, i.e. 100 times smaller. Figure D gives the 
oscillogram for K — 5 — 4 sin 6*28 t. 



Figure B 
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Figure E shows an oscillogram for the same data, but with the 
gains of the first two amplifiers doubled, i.e. K x = 400, K 2 = 400. 
As can be seen from these oscillograms, the output quantity X(t) for 
F (/) == const, remains constant irrespective of the variations in the 
plant gain. 

I doubt that anyone could consider that this system is not equiva¬ 
lent to an adaptive system, merely because it is simple and does not 
contain complex search elements or any computing devices. 
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A Comparison of the Measuring Time in Self-adjusting 

Control Systems 

F. MESCH 


Summary 

It seems impossible to select the ‘best 5 self-adjusting system out of the 
many proposals, but it is possible to compare them from a certain 
point of view. In this paper the point of view is the time required in 
measuring the parameters to be adjusted, considering random input 
signals. Three devices are compared which adjust one parameter of a 
control system in such a way that the R. M. S. error becomes a 
minimum. 

Instead of general derivations the comparison is demonstrated, 
using a very simple control system as an example. In each case the 
variance of the measuring unit output is estimated, this varies statisti¬ 
cally because of the finite averaging time. From the variance the 
necessary time constants of the smoothing filter are determined. 

Results: From the three devices the ‘stepwise’ searching method 
requires long measuring intervals. The method with a ‘periodic 
searching signal’ is better because the search by trial and error is 
replaced by an adjustment directed uniquely to the minimum, but 
the measuring interval is in the same order of magnitude. The method 
with a ‘reference model’, however, has also the advantage of a shorter 
measuring time. 

Sommaire 

S’il est impossible de selectionner le meilleur des systemes auto- 
regleurs proposes, on peut au moins les comparer d’apres un cer¬ 
tain critere. Dans ce rapport, le critere choisi est le temps de mesure 
necessaire des parametres regies, en supposant que les signaux d’entree 
soient ateatoires. Avec ce critere, on compare trois elements qui ont 
tous pour but de regler un parametre dans un systeme de commande 
en minimalisant l’erreur moyenne quadratique. Pour effectuer la 
comparaison, on utilise un systeme de commande tres simple. Dans 
chaque cas, on estime la variance de la sortie de l’^lement de mesure; 
cette variance varie statistiquement en raison du temps fini d’eta- 
blissement de la moyenne. A partir de la variance, on determine les 
constantes de temps n^cessaires du filtre de lissage. 

Resultats de comparaison: La methode de recherche «par paliers» 
demande de longues durees de mesure. La methode avec un «signal 
de recherche p6riodique» est meilleure, parce que le tatonnement est 
remplac6 par un reglage uniquement oriente vers le minimum, mais 
la dur6e de la mesure est du meme ordre de grandeur. La methode 
avecun «modele de reference» a un temps de mesure plus court. 

Zusammenfassung 

Es ist kaum moglich, aus den vielen vorgeschlagenen selbsteinstellen- 
den Systemen das „beste“ auszuwahlen. Moglich ist der Vergleich nach 
bestimmten Gesichtspunkten. In der vorliegenden Arbeit ist dieser 
Gesichtspunkt die Meftzeit, die notig ist, urn bei regellosen Eingangs- 
signalen die einzustellenden Parameter zu messen. Yerglichen werden 
drei Anordnungen, die in einem Regelkreis einen Parameter selbst- 
tatig so einstellen, daft die mittlere quadratische Regelabweichung zu 
einem Minimum wird. 

Statt allgemeiner Ableitungen wird der Vergleich an einem sehr 
einfachen Regelkreis als Beispiel demonstriert. Dazu wird jedesmal die 
Streuung der Ausgangsgrofte der Mefteinrichtung abgeschatzt, die 
wegen der endlichen Meftzeit regellos um den gesuchten Wert schwankt. 
Hieraus ergeben sich die notwendigen Zeitkonstanten des Glattungs- 
filters. 


Ergebnisse: Von den drei Anordnungen erfordert das „schrittweise 
Suchverfahren“ lange Meftzeiten. Das Verfahren mit „periodischem 
Suchsignal“ ist insofern besser, als das probierende Suchen durch eine 
zielrichtige Verstellung des Parameters ersetzt wird; die Meftzeit hat 
aber die gleiche Groftenordnung. Das Verfahren mit einem „Bezugs- 
modell“ dagegen hat zielrichtige Verstellung und kiirzere Meftzeiten. 


Introduction 

During recent years many different systems have been pro¬ 
posed which automatically adjust one or more of their dynamic 
parameters. Since these are always non-linear systems it seems 
impossible to compare the proposals in a general manner and 
to select the ‘best’ of them, because no accepted quality measure 
exists. 

In the presence of random input signals which normally 
occur in control systems, the time needed for the measurement 
of the parameters to be adjusted may, as will be shown, gain 
considerable importance. Some authors did not pay enough 
attention to this point in making their proposals. The measuring 
time will therefore be the point of view from which some typical 
systems are compared. 

A Control System as an Example 

The comparison is demonstrated using the example of a very 
simple control system, as shown in the upper portion of Figure 1 . 
The only free element in it is the gain Kr of the controller. 

The input signal w{t) and the disturbance z(t) are assumed 
to have the power spectral densities S ww — k 2 lco 2 and S zz — 
respectively. The parameters k, k and the gain Ks of the process 
may change in an unpredictable manner. The optimal adjust¬ 
ment of Kr is to be found, using the mean square of the error 

xi(t)= lim xl(i)dt (1) 

r-*oo z i J -t 

as an optimization criterion or performance index. 



Measuring device 


Figure L Stepwise searching method 
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This mean square value may also be calculated in a common 

way from the power spectral density of the error x vl as a mean 
power by r 

*w(0=Jo S XwXw (m)do (2) 

providing stationary input signals and constant parameters, 
brace the parameters, however, are subject to changes, the as¬ 
sumption is made for the following that they are changing slowly 
compared with the relevant time constants of the system; so it 
is still possible to consider the processes as stationary and to 
use eqn (2). 

Applying this to the control system of Figure 1 and intro¬ 
ducing K = Kr ■ Ks, 

Sx„ Xvi (to) S ww (co) + ~ 2 — —2 ■ S zz (co) (3) 

With the assumed spectra S ww and S zz one obtains 


and 


k z (l+K 2 K 2 ) 

x "*" v “ v K 2 + co 2 ’ 


S r ^ v (co) 


x l(t)=Y kZ (-j£+K 2 K 


where a minimum is found at 


K on = - 


(4) 

(5) 


Stepwise Searching Method 

The most obvious idea for the automatic search of this mini , 
mum is to make a computer measure the criterion for various 
values of K successively and then select the best one. Therefore, 
the quantity Q to be measured by the adaptive loop is the mean 
square error itself, giving Q = x w \t), and the corresponding 
measurmg device is shown in the lower portion of Figure 1 The 
integration over an infinite time interval according to eqn (1) is 
impossible, of course; a finite time integration would have to be 
repeated permanently for the self-adjustment. An easier realiz¬ 
able averaging device than an ideal integrator is a continuously 
acting low-pass filter, consisting of a simple lag with time con¬ 
stant T. Because of the finite averaging time, there does not 
occur the desired value Q at the output of the low-pass filter but 
a random variable q(t) fluctuating about the true value Q : 


% (0 — Q + # i(t) 

where q x = q-Q = q-q denotes the fluctuating (a.c.) compo¬ 
nent. The magnitude of the fluctuation obviously depends on the 
filter time constant T. To compare T of different systems, this 
dependence requires a more detailed analysis. 

The magnitude of the fluctuation can be expressed as usual by 
the standard deviation cr, of q. One obtains <r a a either from the 
autocorrelation function 4> aa {r) for r = 0 (see for example 
references 1 and 2) or from the spectrum S, m (oj) by integration 3 
over co. 

Here the latter way is chosen. At first one calculates the 
spectrum S uu (co) behind the squarer 4 * 5 : 

S„ u (®) ~ 2 4> 2 XvjXvi (0) • <5 (co) 


+ 



O^i) * (co — coj) dcOi 


(6a) 


where 0 2 * wXw (0) — (* w 2 (/)) 2 — Q 2 is the mean value of u{t) 
and the integral on the right hand represents the spectrum 
of the fluctuating component. 

For the example this becomes by eqn (3) 


S lu » 


_2%k (l+K 2 k 2 ) 2 1 


K 


4 K 2 +co 2 


(6 b) 


and from the spectral density of the fluctuating component at 
the low-pass filter output 

^Iqq 0°) ~ _|_ (a 2 J.2 ‘ ^1 „„ (Ot) 


one obtains after integration over co 



5 i„0)d® = 


n 2 k 4 (l+K 2 K 2 ) 2 
2K 2 (1 + 2KT) 


As shown below, the smoothing time constant Tis much larger 
than the time constant 1/K of the closed loop, thus KT$> 1; 
t elating the variance u a to the exact value Q, one obtains 


.* rsj 

Q ( KT)* 


(7) 


Because of the non-linear operation of ‘squaring’, the ampli¬ 
tude distribution of q is not normal. However, if T becomes 
sufficiently large and thus <r Q -4. Q, the distribution, due to the 
central limit theorem of probability, approaches a Gaussian one 
with 95 per cent of all values lying within + 2cr. Setting the 
desired accuracy 

P%_ 2a, 

100 Q 

the actual value will be within the error limit ± P * g/100 with a 
95 per cent probability. This results in the smoothing time con¬ 
stant 


T (P%) 2 -K (8a) 

and for the desired accuracy of 5 per cent 

T = 1,600/K (8 b) 

that is, 1,600 times as large as the time constant of the control 
loop. Since several trial steps are usually necessary to reach the 
optimal adjustment, the result is a very large, often intolerable, 
amount of time. 


Sinusoidal Searching Signal 

In optimalizing controls there is a well-known technique of 
measuring the slope of the curve, the extremum of which is 
searched, by using a small periodic perturbation signal and a 
subsequent phase-sensitive detection. Instead of a physical vari¬ 
able, it is also possible of course to perturb a dynamic para¬ 
meter periodically, and from Figure 1 follows the system of 
Figure 2 6 > 7 . In the example K is changed sinusoidally: 

K r = K (14- m 1 • sin co s t) (9) 

The mean square error, obtained as in Figure /, is being multi¬ 
plied by the same searching signal (phase-sensitive detection) 
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and smoothed in low-pass filter 2. The low-pass filter 1 (dotted 
lines) can then be saved, and the following considers only the 
one filter 2. 

As already seen, the measurement of a parameter requires time. 
The periodical perturbation of such a parameter must therefore 
be very slow, which means that co s must be small compared to 
the reciprocal of the time constant of the control loop. This as¬ 
sumption permits one to calculate the mean value u = x w 2 of 
the squaring device output u(t), as on the first page of this paper, 
and to subsequently introduce the sinusoidal parameter per¬ 
turbation (« denotes the averaging over a time interval that is 
small in comparison to the period of the searching signal). Thus 
eqns (4) and (9) give for this example 


This seems at first to be still worse than eqn (7) for the stepwise 
searching method; the reason is that the quantity Q , with ap¬ 
proximately the same o Q , is smaller by the modulation index 
But it must be taken into account that here the accuracy of the 
measurement is not as far as critical; only the sign of q must be 
correct for the parameter adjustment running towards the 
optimum. Hence 2a Q = Q is sufficient (see Figure 3), and with 
the same numerical values as in eqn (lib) one obtains 

^ 8(K 2 k 2 + 1) 2 2,200 

1 ”m?(KV-l) 2 -K _ K ( > 

This is the same order of magnitude as with the stepwise search¬ 
ing method. 


n 


1 


K{l + m x sin co s t) 


+ k 2 K (1 -b m t sin co s t) 


After multiplication by s = m 2 - sin co s t and final smoothing, 
the quantity Q measured by the adaptive loop now becomes, 
with m 1 1, 


Q = x 2 ( t )• m 2 sin (o s t^—k 2 m 1 m 2 ( k 2 K — 


K 


( 10 ) 


the self-adjusting loop being supposed opened behind the measur¬ 
ing unit. As desired, the quantity Q is proportional to the differ¬ 
ential quotient of the criterion x w 2 = f(K ) and zero at the 
minimum K ovt = l//c. At both sides of the extremum, Q has 
different signs and thus behaves like the error of a conventional 
control system. This enables the trial-and-error method to be 
replaced by a methodical adjustment of the parameter, and a 
simple on-off controller is sufficient, as indicated in Figure 2. 


This is an important advantage of the method. However, 
regarding the measuring time it offers no progress. For proving, 
the variance a q of q related to Q is estimated in Appendix I 
to be 


V2( KV + 1) 1 

Q ~(K 2 k 2 — !)•/«! (KTf 


(11a) 


Setting, for instance, K = 2//c, that is twice as large as optimal, 
and m 1 =0-1 corresponding to a perturbation of K by 10 per 
cent, 


a q 24 
_ ±_ _ 

Q ~(KTf 


(lib) 




Figure 3. Amplitude distribution density of the measured variable q 

The modulation index m 1 has a quadratic influence on T, 
and increasing m 1 could save measuring time. The selected value 
m 1 = 0T, however, might be adequate for practical applications; 
otherwise the desired approach to the optimum would be frus¬ 
trated by excessive searching movements. 

At the optimum itself, where K-k = 1 and Q = 0, aJQ and 
T become infinite. One will make a compromise here, too, 
between the measuring time and the accuracy of optimization. 

Model with Parameter-perturbation 

The stepwise searching method (shown previously), measures 
the performance criterion in one system in two successive steps; 
the comparison of both measurements determines the direction 
(and possibly the size) of the next control step. The idea of the 
following method is, in contrast, to measure the criterion simul¬ 
taneously in two almost congruent systems differing only by a 
constant relative perturbation A of the parameter that is to be 
optimized 8 . The difference of both measurements, related to the 
parameter perturbation, gives the difference quotient of the 

7JK + & -K>—;;<K) 

4 K 

as an approximation to the desired differential quotient. Figure 4 
shows the method applied to this example. The model (index M) 
simulates the whole control system, called plant (index A ), and 
receives the same input signals. The mean square errors of both 
systems are subtracted; one common smoothing filter is sufficient 
for the difference e = ua — um = x w A 2 (t) — x w m 2 (0 because of 
the linearity of the subtracting point. The total gain of the model 
loop is denoted by Km — K; the gain of the plant loop is as¬ 
sumed to be perturbed by A • K, thus Ka = K (1 -F A). With 
eqn (4) the difference of both mean squares gives the value Q 
measured by the adaptive loop 

g=^(0-^^(0«|-fe 2 ( K 2 J s:-l)-A (13) 
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neglecting higher powers of A. This equation corresponds com¬ 
pletely to that of (10) for the periodic searching signal; it is 
again a measure for the differential quotient of the criterion as 
a function of K, having the same advantage in respect of a con¬ 
tinuous and methodic parameter control. In addition there is an 
important advantage concerning the measuring time. An evalua¬ 
tion similar to that for the searching method yields the variance 
<y Q of the measured variable q at the low-pass filter output (see 
Appendix II) 

Ttk 2 A [(3KV-1) 2 + 4KT(kV-3) 2 ]* 

S K ’ KT ( 4) 

the adaptive loop being opened again behind the low-pass filter. 
Higher powers of A have been neglected, as well as again the 
terms small with respect to KT. 

Setting for example k • K = 2 as above, this becomes, related 
to the true value Q given by eqn (13) 

^(121 + 4 KTf 
Q ~ 12 KT K J 

Q and a q are proportional to A in a first-order approximation; 
consequently aJQ and thus T do not depend on the magnitude 
of the perturbation. As in the case of the periodic searching 
signal, only the sign of Q is to be determined correctly; again it 
is sufficient to set 2 cr Q = Q and to have a smoothing time con¬ 
stant of „ 

T -K < 16 > 

that is approximately 1/1,000 the value of the other two methods. 
This apparent difference may be explained generally by the fact 
that much more information is incorporated in the self-adjusting 
device within the model requiring a detailed knowledge of the 
plant. The device itself therefore needs to gather less information 
implying a saving in time. 

The short measuring time of devices with a reference model 
can be utilized, not only for the adjustment to an optimum, as 
shown here, but, in general, for the measurement of parameters 
and consequently also for the self-adjustment to a predeterminate 
value 9 ’ 10 . Another advantage is that the normal operating signals 
are sufficient for the self-adjustment without any need of dis¬ 
turbing test signals. 



Plant 


Model 




I Adaptive 
f loop 


Figure 4. Plant with reference model 


The identity of plant and model, excepting the one para¬ 
meter, was postulated above in order to get the desired difference 
signal. However, inequalities of the remaining parameters, and 
also additional disturbing signals not considered in the model, 
would generate additional undesired difference signals and 
pretend a false optimum. This difficulty may sometimes be 
circumvented by tracking the plant parameters that one expects 
to vary, and transferring the changes to the model, using similar 
or other common methods. However, this causes an essential 
complication. 


Appendix I 

In estimating the variance a Q of the measured variable q 
greater omissions are permissible than in determining the true 
value Q, as shown under the heading ‘Sinusoidal Searching 
Signal’. In the output signal u(t) of the squarer, only minor 
components are oscillating periodically, arising from the pre¬ 
ceding modulation of the gain K' = K(1 -{-m 1 • sinco s r) with 
m 1 -4 1; this periodic component can now be neglected. In 
computing Q, the steady component behind the phase-sensitive 
detection, this omission was, of course, impossible because 
otherwise no constant component at all would have come out. 
After the squarer, however, the multiplication of u(t) by s(t) 
— m 2 • sin co s t or modulation with an index 1 is not negligible. 
Therefore eqn (6b) can be used for the spectral density S luu (co) 
of the fluctuating component behind the squarer with invariant 
K, and only the influence of the multiplication by s(t) has to 
be analysed. 

It is practical to start from the autocorrelation function 
(j) vv (t) of the multiplier output v(t) defined as an ensemble 
average. For a given instant t 


CO 



^t + T W[v t ; i> (+t ]di; ( di; 

t+X 


Now v(t) = u(t) • ,?(£). For a certain instant t or (t + t) the 
random variable u{t) forms an ensemble; the periodic signal s(t\ 
in contrast, is a fixed number that can be written before the 
integral sign: 


4 s vv (0 



u t -u t+r -W [ii t ;u t+T ']du t du t+z 


— S,' + T ' (t)j 


The time average of this is 


and with s = m 2 ' sin co s t 

, , , m 2 , / n 

^vv ( T ) = "y * cos 0) s t * </> MM (t) 


The constant component in (j ) uu , multiplied by cos co s r } pro¬ 
duces a ‘deterministic’ error of measurement which is not of 
interest here. Without it, the Fourier transform gives the spec¬ 
trum of the fluctuating component of v(t) 


S !„(©) = 


4 


is lmt K-®)+s lu „K+co)] 
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Inserting eqn (6 b) one obtains for the example 


m 2 2 itk*(l + K 2 K 2 ) 2 


2 X 


1 


+- 


1 


4 K 2 4- (o) s — co) 2 4 K 2 4- ( co s + co) 


The spectrum behind the low-pass filter is 

1 


s lo »= 


1 +co 2 T 2 




and gives after integration and some algebraic manipulations, 
setting again KT > 1, 


m 00 

J 0 


2 KT 


n 2 m\k A (l+K 2 K 2 ) 2 

( w )dco — 2 " T 2 (co 2 +4X 2 ) 


(11a) 


2nk 4 [l + K 2 -K 2 (l + A) 2 f 


and 


X(l+A) 4X 2 (l + A) 2 + co 2 


27tk 4 [l + K 2 K 2 ] 2 


S, (©) = 


(17) 


(18) 


where 


Eqn (20) 


Eqns (17), (18) and (20) for the spectra are inserted into (19). 


After some algebraic manipulations the terms with A 0 and with 
A 1 cancel, and omitting A 3 and higher powers of A 

c , , 7rfc 4 A 2 4 X 2 (K 2 k 2 — 3) 2 + co 2 (3 K 2 k 2 — l) 2 „„ 

S i- (C0) ~~ 2K~‘ (4K 2 ~Vo7? (21) 

The spectrum behind the low-pass filter 

(®) = l + J.2^2 ' S l„. (®) 

is integrated, giving 

nk 2 A [3 X 2 k 2 — l) 2 + 4 XT (X 2 k 2 — 3) 2 ]* 


XT 


(14) 


Earlier in the text a searching signal slow with respect to the cut¬ 
off frequency of the control loop was assumed, i.e. co s <i K. Hence 
the ratio of the variance to the desired value Q given by eqn (10) 
becomes _ „ 

g^ A /2-(xV +1) 1 

Q ~(X 2 K 2 -l)-m 1 (XT)* 

Appendix II 

The spectra of the a.c. components behind the squarer 
become, for the plant and model of Figure 4 and using eqn (6 b), 

S t (co) 

* H A U A \ S 


K 4 K 2 + co 2 

respectively. 

For the spectrum 5 lee (co) of the difference e = ua — um the 
cross power spectra must be considered because of the statistical 
correlation of ua and um' 

S ee (P>) = S u M u M + Su A u A ~ S u M u A ~ ^u A u M 

= S u .+S. a -2Re{S m „} (19) 


4 8 K 

with KTP 1. 
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2 7 ifc 4 (l + k 2 X 2 ) • [1 + k 2 X 2 (1 + A) 2 ] • [4 X 2 (1 + A) + co 2 ] 
X • (1 + A)* • (4 X 2 + co 2 ) • [4 X 2 (1 + A) 2 + co 2 ] 


( 20 ) 


DISCUSSION 


B. Qvarnstrom, Chalmers Institute of Technology, Gibraltargatan 5 R, 
Gothenburg S, Sweden 

In his very interesting paper Mr. Mesch considers an adaptive control 
loop, in which a gain parameter is adjusted by use of a measured 
quantity q (t). This quantity yields in some way or other the informa¬ 
tion about the performance of the main control loop judged on the 
basis of a quadratic criterion. Precisely how the gain parameter will 
be controlled by the quantity q (t) is not discussed by the author, but 
there is no doubt that an adaptive control action can be realized. 


The stepwise searching method suggested incorporates an adaptive 
control of the gain parameter, designed to minimize the mean square 

error Q — x 2 , or, when Q is unavailable, to minimize the quantity 
q (t), which is the best estimation of Q under certain conditions. The 
author has shown in eqns (4) and (5) that Q will reach a minimum for 
a certain value of the gain parameter K , denoted by K op t. This value 
of the gain factor is considered, by definition, to be the best or optimal 
value of the gain. 

The choice of an optimizing criterion is an important one and I 
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would like to know more in detail how the author will justify the 
choice in this case, because there is a slight contradiction in the paper, 
as I see it. On one hand the minimum of the mean square error is ex¬ 
pressed by eqn (4) under the assumption that the gain parameter main¬ 
tains a constant value during the whole averaging or smoothing period. 
On the other hand, the task of the adaptive loop is to continuously 
control the gain parameter, which means that the gain might vary or 
fluctuate if a non-constant gain is preferable; and, this is the point, 
a suitable gain parameter variation may in this case reduce the mean 
square error under the minimum value given by a constant gain. 

The measuring time calculated by the author is a consequence of 
a restriction upon the fluctuation of the measured quantity q it). From 
a practical point of view, I think this restriction might be necessary, 
but I can hardly see any fundamental reason why the fluctuations of 
q (t) should be kept within these or those limits. There is no harm in 
a fluctuating signal, especially not, as I see it, when the variation yields 
information about the signals fed into the system, an information that 
can be used to reduce the mean square error. It is easier for me to 
accept a restriction on the fluctuations of the gain parameter K, be¬ 
cause there must be a number of practical reasons not to allow a wild 
fluctuation of the gain factor in a real plant. 

There is a deep similarity between the stepwise and the sinusoidal 
searching methods. One would expect a smoothing time of the same 
order of magnitude in both cases, as the author already has shown in 
his paper. Now, the function Q is not quite the same in these two 
examples, and I am not sure that the restrictions used therefore are 
equivalent. I would appreciate the author’s comments on this point, 
because I found it interesting that an intentional perturbation did not 
promote the adaptive control possibilities. 

In the third example, the model reference system, the author dem¬ 
onstrates a remarkable scheme permitting the quantity Q to be measur¬ 
ed 1,000 times faster than before. By the use of a model, almost 
identical with the plant, a major part of the statistical fluctuations of 
the squared error is cancelled. In the beginning of the paper the author 
states that the gain of the process Ks may change in an unpredictable 
manner. I cannot see that the model reference system would work 
properly under this condition, because a specific and known gain 
difference between model and plant should be necessary. 

The whole system, plant and model, forms a kind of differentiating 
device. It is well known that devices of that kind are very sensitive to 
disturbances. One consequence of a disturbance other than 2 , that is 
measured and also fed into the model, must be a contribution to the 
fluctuations of the controlled gain parameter K. I feel that in neglecting 
this source of disturbance the author might have overestimated the 
performance of the model reference system. 

Finally, I would like to say that I agree with Mr. Mesch in most 
of his conclusions and I have found his work stimulating and important. 


F. Mesch, in reply 

Answering Professor Qvarnstrom, a quickly-varying gain K would 
mean an essentially and intentionally non-linear system; for Gaussian 
input signals as assumed in the paper, however, it is well known that 
linear filters are optimal in the Wiener sense. Thus linear basic loops 
were considered with slowly-varying parameters. The second com¬ 
ment is quite true; in the first case, the function Q is the error criterion 
to be optimized, and in the two other cases it is the derivative of this 
criterion with respect to the parameter K. I am in complete agreement 
with the third comment. As already pointed out in the oral presentation 
of the paper, the model reference system is sensitive to noise signals 
not fed simultaneously into the model. Therefore the model system 
might be used only as a double model, e.g. on an analogue computer. 
But recently we developed another configuration combining the model 
system with the sinusoidal perturbation system. Preliminary results 
indicate that this more elaborate configuration has a short measuring 
time due to the model, but is also insensitive to non-considered noise, 
due to correlating the output signal with the test signal. 


D. Siljak, Electrotechnical Faculty , University of Belgrade, Macvanska 
8, Belgrade, Yugoslavia 

The comparison of the measuring times of controller gain in various 
first-order self-adjusting systems is straightforward, the results indicat¬ 
ing that the measuring time of the gain to be adjusted is of an essential 
interest. However, the proposed approach, if applied to higher-order 
systems, may suffer from the limitation that the existence of a finite 
optimum value of adjustable gain is assumed. As known 1 , if a control 
system of the second, or higher order, is optimized according to the 
mean-squared error as a performance index, it may happen that only 
the infinite value of the system gain makes the optimum value of the 
chosen index. In order to obtain physically realizable and practically 
applicable values of adjustable system parameters, it is necessary to 
optimize the mentioned performance index subject to the constraints*- 4 . 
The introduction of constraints may change significantly the system 
dynamics, thus affecting the measuring time. It may, therefore, be 
suggested that the author introduce certain constraints into the op¬ 
timization of the controller gain, and make his method applicable to 
more general system configurations. 
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F. Mesch, in reply 

I agree with Dr. Siljak that an optimum must exist if it is to be found 
by the self-adjustment, and that constraints might be important and 
might influence the form of the basic loop. However, the basic loop 
is the same in all three cases; moreover, recent results indicate that 
the model system is quite generally superior to the other two systems, 
independent of the basic loop considered. 

F. B. Tuteur, Department of Engineering & Applied Science, Yale 
University, Conn., U.S.A. 

The startlingly large improvement in measuring time obtained by the 
author with his third system relative to the first two systems appears 
to be due entirely to the fact that the model and system receive precisely 
the same signal and noise. The measurement is therefore essentially a 
noiseless measurement. This is not true of the first two methods con¬ 
sidered. 

It would also be interesting to have the author’s comment on 
whether there is a physical reason why eqn (15) gives a linear short- 
time relation between R.M.S. error and estimation time, whereas the 
more usual relation is that the R.M.S. error varies inversely as the 
square root of the time [as in eqns (7) and (lib) and, for instance, as 
given in the paper by B. Qvarnstrom]. 

F. Mesch, in reply 

I disagree with Professor Tuteur’s first comment. The model system 
is superior to the other two systems even if there is only one signal in 
all cases, without noise. The reason is, as indicated in the paper, that 
in the model there is more a priori information incorporated resulting 
in a shorter time for gathering the rest of information required. The 
simple square-root relation between variance and measuring time 
occurs, generally, only for very long measuring time constants. For 
shorter ones, as obtained with the model, the relationship is more 
involved, as might be seen, e.g., from References 1 and 3 of the paper. 
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0. L. R. Jacobs, Department of Electrical Engineering, Edinburgh Uni¬ 
versity, Edinburgh 9, Scotland 

My question concerns the third system described in the paper; this 
system uses a model with parameter perturbation and is shown in 
Figure 4 of the paper. It can be seen that the random disturbance Z, 
which is partly responsible for errors in the original system, is assumed 
to act on the model as well as on the plant. This implies that disturb¬ 
ance Z can be measured and made available to a controller. 

If it is possible to measure Z, and if the plant is sufficiently well 
understood that it can be modelled as proposed, would it not be 
preferable to control the variable gain K by some direct feedforward 
control from the measured value of Z1 

F. Mesch, in reply 

Dr. Jacobs’ proposal seems to me impractical. It is not sufficient to 
measure only the disturbance Z ; it would be necessary also to measure 
its statistics (power spectrum) and to automatically compute the 
optimum parameter setting. This would be much more complicated. 

G. Rosenau, Deutsche Forschungsanstalt fur Luft- und Raumfahrt , 
Steinhorstwiese 20 a , Braunschweig , Germany 

(1) Mr. Mesch gives the power spectral densities for the input signal 
Sww = k 2 /w 2 and for the disturbance S zz = K 2 k 2 . 

(a) What were the reasons for choosing these power spectra ? 

(b) Did the author investigate other power spectra which would be 
more realistic for applications as, for example, in aeronautics? 

(c) Did the author attempt to generalize his results to the case 
where the input signal w and the disturbance z have different power 
spectra, e.g. having different frequency bands. 

(2) Is it possible to generalize the model reference systems to the 
case where one does not know the disturbance, which will be the 
general case, and where, therefore, it is impossible to introduce the 
disturbance into the model (see Figure 4). 

F. Mesch, in reply 

I disagree with Mr. Rosenau. I would like to know what kind of 
spectra would seem to him more realistic. Command input signals 


usually have spectra decreasing with frequency, and disturbances are 
usually broadband. As the spectra are the same in all three cases, the 
comparison is little affected by their specific form which has been 
chosen, therefore, for computational ease. The second question I have 
already answered in my reply to Professor Qvarnstrom. 


C. M. Woodside, Cambridge University , Cambridge , England 

I wish to suggest an alternative formulation of Mr. Mesch’s model 
approach, which demonstrates its similarity to the step search method. 

He postulates that the random input and noise are available to the 
model. Therefore a tape recording of them can be made. If the search 
proceeds, as in the stepping method with plant alone, and two esti¬ 
mates of mean squared error are made in two consecutive ‘runs’ with 
Kr perturbed by a factor 1 4- A in the second run, then if the tape 
recording is started in the same place at the beginning of each run, 
the first run corresponds with the model in the paper, and the second 
with the plant. The difference between estimates corresponds exactly 
with Q in eqn (15). 

The vast part of the time reduction in obtaining Q, which is still 
approximately 1 / 500 , is then seen to be due to knowing the input and 
disturbance exactly rather than knowing the plant exactly. Knowing 
the plant reduces the time by a further half. 

In comparing sinusoidal and stepping search, it would be more 
instructive to compare the derivative signal in the former with the 
difference between two successive estimates of mean square error in 
the latter, rather than with one estimate of mean square error as in 
the paper. The times required to give 5 per cent accuracy are then 
equal. 

F. Mesch, in reply 

I find Mr. Woodside’s proposal very interesting because it eliminates 
the need of a model, and yet yields a short measuring time. However, 
it assumes that arbitrary (tape recorded) input signals can be applied 
to the system. On the contrary, in the paper it was assumed that only 
normal operating input signals are allowed. I agree, too, that it is more 
correct to consider the difference of two measurements, and indeed 
we have been doing this for some time. 



On Self-adjusting Control Systems Without Test 

Disturbance Signals 

E.P. POPOV, G.M. LOSKUTOV and R.M. YUSUPOV 


Summary 

The report presents and analyses one of the principles for designing 
an adaptive system to control time-varying processes. According to 
this principle, in the processes of self-adjusting, the dynamic character¬ 
istics of the system (plant and controller) and the effect of some un¬ 
controlled external disturbances are taken into consideration. This 
controlled process is close, on the whole, to the predetermined (ref¬ 
erence) behaviour in adjusting controller parameters. A high-speed 
C0m puter, either a digital or an electronic analogue one, is supposed to 
be included in the loop of the adaptive control, depending upon the 
method of the process identification. Dynamic characteristics of the sys¬ 
tem are found by differential equations with time-constant coefficients, 
by means of which real and reference processes are approximated. 

In estimating reference values of controller coefficients, the sum of 
the squares of real coefficient value deviations from the reference ones 
is minimized. Mechanization problems of computing the above co¬ 
efficients by definite servos are discussed. The report deals with some 
particular applications of the given method for designing such an 
adaptive control system. 

Sommaire 

Ce rapport met en evidence et etudie l’un des principes permettant 
d’elaborer un systeme adaptatif destine a commander des processus 
eyoluant dans le temps. Conformement a ce principe, on prend en con¬ 
sideration, pour Fauto-adaptation les caracteristiques dynamiques 
du systeme (installation et commande) et l’effet de quelques pertur¬ 
bations externes incontrolees. Le processus commande est, d’une 
maniere globale, rendu proche du comportement predetermine (refe¬ 
rence) par reglage des parametres de la commande. On suppose qu’une 
calculatrice rapide du type numerique ou analogique, est insere dans 
la boucle de la commande adaptative, compte tenu de la methode 
d’identification utilisee. Les caracteristiques dynamiques du systeme 
sont donnees par des equations differentielles a coefficients independants 
du temps, equations qui permettent d’approximer les processus reels 
comme les processus de reference. Lors de Festimation des valeurs de 
reference des coefficients du regulateur, on minimalise la somme des 
carres des ecarts des coefficients reels par rapport aux coefficients de 
reference. Les problemes li6s a la mecanisation du calcul de ces coeffi¬ 
cients a l’aide de servo-mecanismes sont passes en revue. Le rapport 
s’acheve par la description de quelques applications particulieres de la 
methode proposee a Felaboration de commandes adaptatives de ce type. 

Zusammenfassung 

Dieser Beitrag behandelt ein Bauprinzip fur anpassende Regelung zeit- 
veranderlicher Prozesse. Hierbei werden fur die Selbstanpassung die 
dynamischen Eigenschaften des Systems (Strecke und Regler) und die 
Auswirkungen einiger ungeregelter aufierer Storeinflusse beriicksich- 
tigt. Bei der Einstellung der Regelparameter entspricht dieser Regel- 
vorgang im grofien ganzen dem Sollverhalten. Je nach der Art der 
Regelstrecke ist der Schnellrechner im selbstanpassenden Regelkreis 
entweder digital oder elektronisch analog. Die dynamischen Eigen¬ 
schaften des Systems ergeben sich aus Differentialgleichungen mit 
Zeitkonstanten-Koeffizienten, durch welche die wirklichen den be- 
zogenen Prozessen angenahert werden. 

. Die Summe der quadrierten Abweichungen dieser beiden Prozesse 
wird zur Vorausschatzung der Bezugswerte der Reglereinstellungen 


minimiert. Es wird untersucht, wie diese Koeffiizenten durch vorbe- 
stimmte Steuerungssysteme mechanisiert werden konnen. Der Beitrag 
behandelt einige Anwendungen dieser Methode fur den Entwurf eines 
solchen anpassenden Regelsystems. 


Statement of the Problem 

In this paper, the term ‘self-adjusting control system’ means a 
system which performs the following three operations: 

(1) Measures by means of automatic search or computes 
from the results of measurements the dynamic characteristics 
of the system, and possibly the characteristics of the disturbances 
as well. 

(2) On the basis of this or that criterion defines the controller 
setting, parameters or structure needed for calibration (or opti¬ 
mization). 

(3) Realizes the resultant controller structure, parameter or 
setting values. 

Many studies of the theory and practice of self-adjusting 
control systems for stationary controlled plants have so far- 
appeared in the world literature. There have also been con¬ 
tributions on self-adjusting of quasi-stationary systems. But there 
is almost a complete lack of contributions dealing more or less 
specifically with problems of synthesis and analysis of self-adjust¬ 
ing control systems for essentially non-stationary controlled 
plants. Moreover, as far as the authors are aware, even in the 
case of stationary and quasi-stationary systems, the process of 
self-adjustment is frequently effected solely on the basis of an 
analysis of the dynamic characteristics of the system, without 
taking into account the unmeasured external disturbances acting 
upon the controlled plant. At the same time it is obvious that 
external disturbance, besides the dynamic characteristics of the 
system, determines the quality of the control process. 

Another drawback of many of the self-adjusting systems in 
existence and proposed in the literature is the need to use 
special test signals to check the dynamic characteristics of the 
system. 

This paper proposes, and attempts to validate, one of the 
possible principles for the creation of a self-adjusting control 
system for a particular class of non-stationary controlled plants. 

The main advantage of the principle in question is the 
opportunity it provides to take account of both internal 
(system parameters) and external (harmful and controlling 
disturbances) conditions of operation of the system. In contrast 
to the self-adjusting systems known, a system created in accord¬ 
ance with the principle proposed will make it possible to obtain 
automatically the fullest possible information about the process 
under control without the use of test signals. 
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ON SELF-ADJUSTING CONTROL SYSTEMS 

For the operation of a self-adjusting control system created 
on the basis of the principle proposed, a mathematical model 
of a reference (calculated) control system must be constructed. 

A ‘reference system’ is understood to be a system the controller 
of which is designed in accordance with the requirements on 
the quality of the control process, with the assumption that the 
mode of variation in time of the system’s parameters as well as 
the disturbance effects are known. 

The structure of the mathematical approximation of the real 
process is selected to match that of the mathematical model of 
the reference process. The self-adjusting system operates in such 
a way as to ensure continuous identity between the mathematical 
approximation of the real process and the model of the reference 
system. In this connection, the problem is posed of making the 
mathematical approximation of the real process as close as 
possible to the model of the reference process. 

Without loss of generality, the case of control of only one 
variable is considered, which is denoted by x, and the correspond¬ 
ing reference differential equation is written in the form 

n — 1 m 

x ( E n) + I a f(o4°= I bl(i)Sf (1) 

£ = 0 i-0 

The real process is approximated by a linear differential equation 
of the same structure: 

n-l m 

* W +E fli(0* w =E*m(0/ <0 (2) 

i-0 i = 0 

t = t 09 xW(t 0 ) = x<& 0 = 0,1,..... ft — 1) 

The operation of the proposed self-adjusting control system 
will be examined in accordance with the sequence of the process 
of self-adjustment, indicated at the beginning of the definition. 

General Case of Determination of the Dynamic Characteristics 
of a System 

In order to create an engineering method of determining the 
dynamic characteristics of non-stationary systems in the construc¬ 
tion of a self-adjusting control system, this paper proposes the 
use of the methods of stationary systems. For this purpose, the 
non-stationary system (1) is replaced by an equivalent system 
with piecewise-constant coefficients. (The methods of stationary 
systems are used in the intervals of constancy of the coefficients.) 
The transfer from a system with variable coefficients to one with 
piecewise-constant coefficients is effected on the basis of a 
theorem which can be formulated with the assistance of a 
number of the propositions of the theory of ordinary differential 
equations. In accordance with this theorem, the solution of a 
differential equation of form (1) with piecewise-continuous 
coefficients (a finite number of discontinuities of the first kind 
is assumed) can be obtained with any degree of accuracy in a 
preset finite interval (t 0 , T 0 ) by breaking down the latter into 
a finite number of sub-intervals (fe, tK+ i) and replacement of 
the variable coefficients within each sub-interval by constants, 
equal to any values of the corresponding coefficients inside or 
on the boundaries of the sub-intervals under consideration. 
In the general case, it is expedient to effect the breakdown 
process by the method of multiple iteration of solutions on a 
high-speed computer. 


WITHOUT TEST DISTURBANCE SIGNALS 

Let the differential equation with variable coefficients (1) 
be approximated by an equation with piecewise-constant 
coefficients. 

Then, for t e (t K , ta-i), one may write 

n-l nt 

4 n) + E «= I *&/ (i> (3) 

i—0 i —0 

In accordance with differential equation (3), the real process 
is approximated by the equation 

n-l m 

x w + £ ««* (0 = E b iK f li) (4) 

i—0 i = 0 

As dynamic characteristics of the system at the first stage 
of operation of the self-adjusting system in each interval 
(tR 9 tx+ i), the coefficients a t K (z = 0, 1, ..., n — 1), biK (i = 0, 
1,..., m) are defined. 

The simplest way to define these coefficients lies in defining 
the values of x and/and their corresponding derivatives at the 
points tR = Tl, t 2 , ..., r s = fa+i — A t. 

By substituting these values into eqn(4), one obtains for 
each interval (Jr, t k-fi) a system of S algebraic dissimilar 
equations for defining the searched coefficients. 

In practice it is not always possible to measure the disturbing 
effect / and its derivatives. Therefore, in the general case, the 
above-mentioned method of defining the coefficients aw and bw 
cannot be directly employed. 

This difficulty may be avoided in the following way. The real 
process is approximated, not by differential eqn (4), but by a 
differential equation of the form 

n-l m __ 

x ( ">+ E L* (0 = E &«/i° (5) 

i = 0 i — O 

In eqn (5) the disturbing effect and its corresponding deriva¬ 
tives are taken to equal the reference values. This avoids 
the need to measure the real disturbance /, and makes it 
possible to use the above-mentioned means of defining the 
coefficients of the differential equation approximating the real 
control process. The non-agreement of the real disturbances 
with the reference ones are taken into account through the 
coefficients aw and bw- Therefore dashes are placed over them. 

In the general case X^ s|= (/ = 0, 1, ..., ft) i.e., there 

is an approximation error. In view of this, in the transfer from 
eqn (4) to eqn (5), it is necessary to evaluate the maximum 
possible value of this approximation error, using for this 
purpose the assumed values of the limits of variation of 
disturbance /. 

If for some class of controlled plants it can be assumed that 
in the process of operation only the scale of the disturbance 
changes, i.e., the equality 

f (t)= C K f E (t), te(t K ,t K+i ) (6) 

where Ck is the random scale of disturbance, is satisfied, then 
the approximation error is absent, and the connection of the 
coefficients of eqns (4) and (5) is expressed by the equalities: 

a iK = a iK (i — 0, 1,..., ft —1) 

( 7 ) 

b iK = C K b iK (i — 0,1,..., m) 

Equation (5) is used (henceforward, to simplify the notation, 
the dashes over the coefficients and the variable x are dropped) 
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for definition of the coefficients a t K and b t K. It is assumed that 
measurements x, x ',..., *<”> are performed at the points 
tji = Tj, r 2 , ..., t s = te+i - At. 

The values of are known. Then, for the 

definition of (n + m + 1) desired coefficients in each interval 
(Jk, tx+\) one obtains the following system of S algebraic 
equations, which will be written in abbreviated form thus: 


' X /i° Oj) b ih 

i = 0 


0 = 1,2, ...,S) 


It is not always expedient to solve directly system (8) for 
S wr + 72 + 1, since, on account of the existence of measuring 
instrument errors and random high-frequency control process 
oscillations, the accuracy of definition of the coefficients will 
be very low. Moreover, for the same reasons, system (8) may 
be altogether incompatible. 

To eliminate the case of incompatibility and to increase the 
accuracy of definition of the searched coefficients the method of 
least squares is employed 1 - 2 . In so doing, the problem of 
approximation is also solved. When utilizing this method, 
it is expedient to take S > m + n + 1. 

Using the method of least squares, the coefficients a iK , b iK 
are defined, minimizing according to these coefficients the 
function s 

L = (*/)■£/ 

where 


A/ Z x( } ( T j) a iK — Z /e > (U) b iK + x (M) (t •) 
i = 0 i = o 

is the disagreement, and p (t ; -) are weight coefficients which 
define the value of each measurement and, accordingly, of 
each of the equations of system (8). 

The necessary condition of the minimum of function L is the 
equality to zero of its first-order partial derivatives according to 
ciiK and biK- Having computed the partial derivatives and 
equated them to zero, one obtains an already compatible 
system of m + n + 1 linear algebraic equations for the defini¬ 
tion of m + n + 1 coefficients: 

s QjL 

^=2>(v)i,^-0(i-0,l. n-l) 

0L i, 0L- ( 9 ) 

db^ =p (li) L jQ^r K =o(i=o,i,...,m) 

Solving system (9) by known methods, one obtains the 
values of a iK and b{K . 

In certain cases the process of control at intervals may be 
approximated by a differential equation of the form 


real coefficients b t K and real disturbances / will be taken into 
account in the system via the values of the coefficients a t K- 
System (11) will be the initial algebraic system for definition of 
the coefficients a iK : 

n-l 

X X (i) (t,-) a iK = q> EK (tj) - X (n) (ij) (j = 1,2,. .., S) (11) 

1 = 0 

For definition of the searched coefficients a f K by the method 
of least squares, one minimizes the function 


where 


L i= X 

j =l 

L j = X x(0 (?j) a iK + (tj)~ (p E K (U) 

1=0 J 


Using the necessary condition of the existence of a minimum 
of function (12) for the definition of n 9 coefficients (i = 0, 

1), one obtains a system of n algebraic equations: 


SL 1 _ V /Nr dL J 

nn... X P(. T )) L jftZ 


j= 1 


0 (i=0,l.n-l) 


xW + X a iK x< 

i=o 


= ( Pek (0 


i = 0 

This coarser approximation will make it possible to reduce 
computing time considerably by a reduction of the quantity of 
searched coefficients; in the given case only the coefficients a iK 
are desired. 

In the given approximation the deviations of the values of 


All the above discussion and the operations were performed 
on the assumption that the values of the control variable 
and the necessary quantity of derivatives at the moments 
of time of interest are available. In practice, however, one is 
usually limited to second-order derivatives. 

In a number of cases real high-order systems may be 
approximated by second-order differential equations, preserving 
the description of their main dynamic properties. But even in 
the case of more complex high-order systems it is possible to 
suggest a number of algorithms for defining the searched 
coefficients, given the existence of a limited quantity of deriva¬ 
tives, some of which are as follows: 

(a) Derivatives of higher orders of the control variable can 
be calculated with the assistance of a digital computer on the 
basis of the Lagrange and Newton interpolation formulae or 
according to the formulae of quadratic interpolation (method 
of least squares). 

(b) If one integrates each term of eqns (5) and (10) n - q 

times, where q is the order of the senior derivative of the control 
variable, which one can measure in a system with the requisite 
accuracy, then, taking the limits of integration tK,Tj(j= 1, 
2, ..., S ), one obtains the integral forms of eqns (8) and (11) 
respectively. If reference values are given to the magnitudes 
x ( n *) (tii), x ( tR ), •.., * ( n ~ Qf+1 ) (t£) in these equations, 

then for defining the coefficients (i = 0, 1, ..., n — 1) and 
biK (i = 0, 1, ..., rri) it is sufficient to measure the derivatives 
to the #th order. 

(c) Practically all existing controlled plants and control 
systems can be described by a set of differential equations, each 
of which characterizes one degree of freedom of movement and 
therefore has an order no higher than second. 

(d) Sometimes, to reduce the order of the derivatives required 
for measurement, one may also take advantage of a number of 
coarse assumptions in relation to the terms of eqns (5) and (10), 
which contain derivatives of high orders. 

For example, in these equations the values of the derivatives 
x( n \ x^-^ 1 ) can be assumed equal to the reference values. 




ON SELF-ADJUSTING CONTROL SYSTEMS WITHOUT TEST DISTURBANCE SIGNALS 


( e ) The coefficients of approximating eqns (5) and (10) can 
be defined without any recourse to algebraic systems (8) and (11), 
if one uses the following method 5 . 

Let the composition of the control system include an analogue 
simulator, on which is set up a differential equation of form (5) 
or (10). In this simulator there is a controlling device, which 
provides an opportunity to effect variation of coefficients a t x 
and biK in a certain way. 

The control system memorizes the curve of the real process 
in the interval (te, tx+ i — A/), and selection of the coefficients 
OiK and b t K is performed on the simulator in such a way as to 
bring together in a certain sense the real process and the solution 
of the equation set up on the simulator. 

When the quantitative value of the proximity evaluation 
reaches the predetermined value, the magnitudes of coefficients 
diK and ^k, are fixed and extracted for subsequent employment 
in the self-adjusting control system. Obviously the simulator 
operation time scale must be many times less than the real time 
scale of the system. Only under this condition can the requisite 
high speed of self-adjustment be achieved. Practically any time 
scale may be realized with the assistance of analogue computing 
techniques. 


Automatic Synthesis of Controller Parameters 


For the operation of the majority of self-adjusting systems, 
the system operation quality criterion is set in advance. For 
systems constructed on the basis of the proposed principle, it 
is generally expedient to use as the criterion the expression 


n-1 rn 

M— £ (a iK -af K ) 2 + £ ( b lK -bf K f (14) 

i=0 i=0 

This criterion generalizes both methods of approximation of 
the real control process expounded above. 

To simplify subsequent operations, the following notations 
are introduced. 


^ 0 K~ a nK.l — •••? &mK &m + n,K 

Expression (14) can then be rewritten in the form 


Yj ( a iK~~ a fK) 2 l 

i = 0 


n n = 


n + m for (5) 
n —1 for (10) 


(15) 


In each interval (fe, t K + 1 ) the adjustable parameters are so 
selected as to bring expression (15) to the minimum. The ideal, 
i.e., most favourable, case would be one when M would reach 
zero as the result of selection of the adjustable parameters. This 
is not always possible, however. In the first place, not all the 
coefficients a iK (j = 0, 1, ..., n 0 ) are controllable. Second, in 
multi-loop non-autonomous systems even the values of the 
controllable coefficients cannot all be tuned up to the reference 
values simultaneously, since the relationship of the coefficients 
tH to the adjustable parameters, although usually linear, is 
nevertheless arbitrary with respect to the quantity of adjustable 
parameters, the sign and the coefficients with which these 
parameters enter into expressions for 

The second difficulty may be avoided by means of successful 
selection of the reference system or by complete disconnection of 
the loops (channels) of control from the main variables, i.e., by 
satisfying the conditions of autonomy. 


It is assumed that all the coefficients a t (i = 0, 1, ..., n 0 ) are 
controllable (in practice the values of uncontrollable coefficients 
may be reckoned to be reference values). Then, for the coeffi¬ 
cients Oi one may write 

a~a l (K t ,K 2 , T u T 2 ,.... T q ; l u l 2 ,..., l r ) 
(i=0,l. n 0 ) 

where K x , K 2 ,..., K p are the gains of the controlled plant; 
T x , T 2 , ...,Tg are the time constants of the controlled plant and 
the controller, and l v 4, l r are the gains of the controller 
(adjustable parameters). 

Since the coefficients a t usually depend on the adjustable 
parameters linearly, one may write 

a t = £ Hijlj+Vi (i=0,1,...,n 0 ) (16) 

i=i 

where 

->K p ; T u T 2 ,.... Tg); 

Vi=v J (iC 1 , ...,K P ; T x , T 2 ,T q ) 

Using the necessary condition for the existence of a minimum 
of function M , one obtains the following algebraic system for 
determination of the setting values l l9 / 2 , l r 


no 


£[««(/l.J 2,- 

i = 0 


i\ ~E 1 fi a iK.(h> ‘**5 K) _ a 

l r )-a iK \ - Wj --0 


0 = 1,2,..., r) 


(17) 


It is assumed that when the system is in operation, the 
adjustable parameter values only change in accordance with 
their computed values, i.e., at any moment of time one knows 
the magnitudes of l l9 / 2 , / r . Then, for the interval (t K , t K -p 

until the moment of correction of the adjustable parameters in 
accordance with expression (16), one can write: 


a iK = £ V-ijK Lk-i + v ;k C 18 ) 

;=i 

From system (18) one may determme the magnitudes of 
jiiijK and v iK (i = 0,1,..., n 0 ; j = 1, 2,..., r) since the values 
of a iK (i = 0, 1,..., no) and l s , K -i U =1,2,..., r) are known. 

Taking into account eqn (16), after substitution of the 
values of and v ik the algebraic system (17) for defining 
l X K, 1 2 k, /^takes the form 


no V / r \ 

£ £ HiJKljK + ViK -4 UkjK = 0 (19) 

i=o L\j = i / J 


0* = l,2,..„r) 


Realization of Adjustable Parameters 

Block-circuit with a Self-adjusting System using a Digital Computer 

The duration of the intervals of constancy of the coefficients 
of reference eqn (3), when a digital computer is used in the 
control system, must satisfy correlation 

t K+ i-tK=T x + T 2 +T 3 +At (20) 

where T x = At (S — 1) is the time required to carry out measure¬ 
ments; T 2 = N/n Q is the time required for the computations; 
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r 3 is the time of actuator operation; 0 < Ar < tx+i — tx\ 
At = r j+1 — tj is the period of measurements (j= 1,2,..., S ); 
n 0 is the computer speed of action, and N is the number of 
operations required to define coefficients (j = 1, 2,..., r). 

It is obvious that to ensure better operation of the self- 
adjusting system, it is necessary to reduce as much as possible 
the magnitude T = 7^ + 72 +TV 

Now the opportunities for reducing the time r 3 are dealt 
with. This question is directly linked with the choice of the 
actuator. Electromechanical servosystems with a considerable 
time constant are usually employed as actuators at the present 
time. But it turns out that it is possible to suggest a number of 
purely circuit variants of the change of transfer functions or 
of gains of the correcting devices (regulators) of the system. 



These inertia-less actuators are termed ‘static 5 . It is particularly 
advantageous to produce static actuators with the aid of non¬ 
linear resistors (varistors), valves with variable gains (varimu), 
electronic multipliers, etc. 

Consider, for example, one of the variants of a static 
actuator based on an electronic multiplier. Let the mode of 
control have the form r 

y = I ljX U) 

j =1 

and let the/th adjustable parameter have the value l } ° at moment 
t 0 (start of operation of the system). While the system 
operates in accordance with the signals of the computer, the 
value l } is constantly being corrected. 

Thus, at the end of the interval (fe, te- 1 - 1 ) one has 

Ok = 0° Ox 

T= E l°x U) + £ Al jK x U) (21) 

j=i j =i 

Obviously each addend in the right-hand side of expression 
(21) can be instrumented with the aid of the circuit in Figure 1. 


The following are self-adjusting system computer operating 
algorithms: when the real process is approximated by 
differential eqns (5), the algebraic systems (9), (18), and (19); 
when the real process is approximated by differential eqns (10), 
the algebraic systems (13), (18), and (19). 

It is obvious that in the general case it is more convenient 
to solve the problem of self-adjustment according to the proposed 
principle with the aid of a high-speed digital computer. It can 
be specialized for solving systems of algebraic equations. 
Figure 2 shows the block diagram of a self-adjusting system 
with a digital computer. 

Some Particular Cases 

In the preceding sections the proposed principle for creating 
a self-adjusting control system for non-stationary plants was 
expounded in general form. In practice, one may naturally 
encounter cases when the given principle can be used in more 
simplified variants. Several such opportunities are considered. 

(1) Obviously, the entire theory expounded above can be 
applied fully to stationary and quasi-stationary systems, which 
are particular instances of non-stationary systems. In this case 
the durations of the intervals of constancy of the coefficients 
0 tR , to- j) equal, for stationary systems 

K=0, t K+1 — t K —t 1 — t 0 — T 0 ~t 0 ( 22 ) 

for quasi-stationary systems 

*k+i — t K >&t p ( 23 ) 

where t Q is the control time (duration of the transient process). 

As can be seen from relations (22) and (23), in stationary and 
quasi-stationary systems one is less rigidly confined to the time 
of analysis of the real process and synthesis of controller para¬ 
meters. It is therefore possible to define coefficients a t K and b iK 
more accurately and to use criteria which reduce the self¬ 
adjustment process speed, but make it possible to increase 
the accuracy of operation of the system. Among such criteria 
one may cite, in particular, the integral criteria for the evaluation 
of the quality of a transient process 3 . 

For stationary and quasi-stationary systems the problem of 
self-adjustment in accordance with the principle proposed above 
may be solved as a problem of the change in position of the 
roots of the transfer function of a closed system, i.e., the self¬ 
adjustment problem may be solved in accordance with the 
requirements of the root-locus method, which is extensively 



450 





















ON SELF-ADJUSTING CONTROL SYSTEMS WITHOUT TEST DISTURBANCE SIGNALS 


employed in automatic control theory. A feature of the use of the 
proposition of the root-locus method in accordance with the 
principle under consideration is that the zeros and poles defined 
by the coefficients am and bm are fictions since they not only 
depend on the parameters of the controlled plant and controller, 
but also depend on real disturbances as well. 

(2) In practice, one may encounter cases when a controller 
is required to ensure only the stability of a system in the course 
of operation. As is known, the stability of linear stationary 
systems is determined by the coefficients of the characteristic 
equation. This proposition is also valid for certain quasi¬ 
stationary systems (method of frozen coefficients). 

Therefore to solve the problem posed (the provision of 
stability), the control system must define the actual values of 
the coefficients of the left-hand side of the differential equation 
of the system and must set on the controller such gains 
as will satisfy the conditions of stability, for example the 
conditions of the Hurwitz algebraic criterion. On the assump¬ 
tion that disturbance / is constant in the interval Qr, te+i) the 
coefficients of the characteristic equation of the system on this 
interval are determined in the following way. 

The differential equation of the system for t e (Jr, te+i) is 
written in the form 

* (n) + Y a iK^ l)== ^K 
£ = 0 

where Fr is in the general case the unknown right-hand side, 
constant for t e (t K , ta-i). The algebraic system for determining 
the described coefficients will then be written thus: 

x (B) (^)+ X x w (Tj)a iK =F K (.7 = 1,2,S) (24) 

i — 0 

Since Fr is unknown, but is constant in the interval (Jr, fe+i) it 
is eliminated with the assistance of one of the equations of 
system (24). For this purpose one uses the equation 

* ( %,) + "z x (i) OO a iK =F K (l<,l<S) 

i = 0 

After eliminating Fr one has: 


"l [x< ; >(T,)-( t,)] a iK = — [x ( "> (Zj)-x<") (t ; )] 

0=1,2. + (25) 

By resolving system (25) directly with S — n + 1, or by the least- 
squares method with S> n + 1, one determines the coefficients 
a . K (/ = 0, 1, ..n — 1), the values of which are used, if the need 
arises, for synthesis of the values of the controller parameters 
which ensure the stability of the system. 

Conclusion 

The paper has expounded only the basis of the proposed 
principle for the construction of a self-adjusting control system 
in general form and in certain particular cases. Studies are under 
way on problems connected with the approximation of differen¬ 
tial equations with essentially variable coefficients by differential 
equations with piecewise-constant coefficients, with the selection 
of the type of computer to operate in the self-adjustment loop, 
with the dynamic precision of the self-adjusting system, etc. The 
investigations which have been made allow one to hope that the 
use of the principle expounded in this paper for the construction 
of self-adjusting control systems will prove extremely effective in 
many cases when it is expedient to use the natural oscillations 
of the system, without introducing test disturbance signals. 
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DISCUSSION 


K. S. P. Kumar, Control & Information Systems Lab., School of Elec¬ 
trical Engineering, Purdue University, Lafayette, Indiana, U.S.A. 

The authors have replaced a non-stationary system with an equivalent 
system with piece-wise constant coefficients. This is useful if the interval 
over which this approximation is valid is longer than the computation 
time needed to identify the coefficients. It would be interesting to hear 
of any experimental results on this. 

Under the assumption of constant or piece-wise constant coeffi¬ 
cients, an alternative procedure that is computationally very efficient 
and conceptually simple is presented below. The method consists in 
viewing the identification as a multi-point boundary value problem. 
A scalar example is given for illustrative purposes. Generalization to 
the vector-matrix case is very straightforward. The method does not 
need any test signals and derivatives of the input. Furthermore, the 
procedure is unchanged even if the system is non-linear. 

Let the control system be described by 

x=fx + gu (1) 

where u is the control variable and/, g are unknown constants. As / 
and g are constants, describe them by 


/= 0 ( 2 ) 

g== 0 (3) 

Make measurements on u and x over a finite time interval, the measure¬ 
ments on x serving as boundary values to solve the system of eqns (1) 
to (3). The method used to solve this boundary value problem is the 
quasi-linearization method 1 . The method has several advantages, chief 
among which is quadratic convergence. If, in addition to the coeffi¬ 
cients, the order of the differential equation of the system is also un¬ 
known, the above method can still be successfully used to identify and 
control the system. The method can also be used for combined 
identification and control. These and other problems have been studied 
at Purdue University with great success and they will be reported very 
soon. 

Reference 

1 Kalaba, R. On non-linear differential equations, the maximum 
operation, and monotone convergence. J. Math. Mech. 8 (1959) 
519 
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E. P. Popov, in reply 

The method suggested by Dr. Kumar is close to that described in our 
paper. We are now working on problems of identification of non¬ 
linear systems. 

S. S. L. Chang, College of Engineering, State University of New York , 
Long Island, N.Y., US.A. 

I have two particular questions to ask: 

(1) In the paper the set of coefficients aw and bw are assumed to be 
independent for each sub-interval K. Sometimes the unknown disturb¬ 
ances are quite considerable and the coefficients aw and bw cannot be 
accurately determined for each sub-interval as the authors proposed. 
An alternative is to use the newly measured data to modify previous 
values of aw and bw. Have the authors studied this possibility and, 
if so, have they any comments to make ? 

(2) Will the authors please comment on the relative merits of their 
proposed method versus the dual control method of Professor Feld- 
baum? 

E. P. Popov, in reply 

Professor Chang suggests, as we understand it, that coefficients aw 
and biic should not be determined directly on each sub-interval far, 
tK- x), but that their previous values a i K - x and b ifK - x should be 
clarified by processing the new information on the interval (t K> t K -f), 
i.e., the quantities Saw Abac should be determined, in order to use 
them subsequently for determining the coefficients 

a iK — a i, K- 1 + ^ a iK 
biK~bi, K-l+&b iK 

We have considered such algorithms. It proved difficult to use them 
because of the impermissibly high values of the absolute errors of 
quantities A aw and Abac due to the approximation, and also the 
existence of measuring and computing errors when the values of the 
quantities Aat K and A bi K themselves are quite small. 

As far as comparison of our adaptive system with dual control 
systems is concerned, it must be said that in our opinion any adaptive 
system, including ours, is a dual-control system. 

F. B. Tuteur, Yale University Department of Engineering and Applied 
Science, New Haven 11, Connecticut, U.S.A. 

(1) The authors propose identification of parameters by the use of 
high-order derivatives of the measured output quantities. However, 
it is well known in practice that derivatives of almost any order are 
almost impossible to obtain with any accuracy from measured data, 
because the process of differentiation simply amplifies the noise. Thus 


this method does not seem very practical. Use of a digital computer 
is of no help in this respect either. On the other hand, the proposal to 
substitute high-order integration would probably suffer from the dis¬ 
advantage that important variations of the signal would be smoothed 
out by the integration process. The authors’ comments on this point 
would be appreciated. 

(2) Since the authors propose to treat a non-stationary system by 
breaking it up into a sequence of piece-wise stationary systems, the 
important question arises regarding how large the true ‘interval’ tic, 
tic+x should be made. If it is too short eqn (20) of the paper is not satis¬ 
fied, and if it is too long the system cannot be considered to be station¬ 
ary. In this connection it should be pointed out that the time required 
to identify a parameter is, in general, several orders of magnitude 
greater than the significant system time constants L 2 > 3. it would 
seem, therefore, that the method could only be used for very slowly- 
varying systems. 
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E. P. Popov, in reply 

We are in full agreement with Professor Tuteur’s comments on the 
difficulties which arise because of the need to have derivatives of a 
controlled variable of high order. We have studied, and are studying, 
this problem. 

Some algorithms of the solution of the problem of identification 
with only a limited number of derivatives are presented in the paper. 
They had to be dealt with very briefly because of lack of time. 

Greatest attention should be paid to the algorithms obtained on 
the basis of multiple integration of differential equations, each of 
which characterizes one degree of freedom, i.e. has an order not higher 
than second. 

We can reply to Professor Tuteur’s second comment as follows. 
For specific systems, selection of the length of sub-interval [/&, tj c +J 
was performed on an analogue computer, proceeding from the require¬ 
ment of the specified accuracy of approximation of the solution of the 
differential equation with variable coefficients describing the system 
by solving an equation with constant coefficients. 

The coefficients aw and bw were determined on the digital and on 
the analogue computer from the results of measurements at intervals, 
the length of which was less than the essential time constants of the 
system. 
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Summary 

The paper deals with the question of the use of self-learning processes 
in adaptive systems with least initial information. The mathematical 
expression, based on the phenomenon of entropy by which the 
dynamics of the learning and self-learning processes are experimentally 
determined, is given in the paper. Also introduced is the phenome¬ 
non of rate of learning which can give the value of the processes. 
Some concrete examples of the learning of automata and some 
living beings are discussed. In the paper are given some experimental 
data of the results of the self-learning system making the automatic 
synthesis of the corrective device. According to the experimental 
data the processes of searching can be significantly lessened by using 
the methods of learning of the adaptive system. The proposed 
mathematical expression can be used for the description of different 
learning processes and makes it possible to achieve some analogies 
between them. Experiments connected with the self-learning of the 
adaptive system were carried out on the special electronic machine 
of th'e Institute of Electronics, Automatics and Telemechanics of the 
Academy of Sciences of Georgia. 

Sommaire 

Cette note concerne P utilisation des processus a auto-apprentissage 
dans les systemes adaptatifs disposant d’un minimum d’information 
initiate. Elle donne, en se fondant sur l’entropie, l’expression mathe- 
matique a Paide de laquelle la dynamique des processus d’appren¬ 
tissage et d’auto-apprentissage peut etre etudiee. Elle introduit la 
notion de taux d’apprentissage, qui peut caracteriser la valeur de ces 
processus. Elle donne quelques exemples concrets d’apprentissage 
dans les automates et chez quelques etres vivants, ainsi que des 
donnees expdrimentales relatives aux r6sultats obtenus sur des 
systemes & auto-apprentissage r^alisant la synthese automatique de 
systemes correcteurs. D’apres ces donnees expdrimentales, il apparait 
que le processus de recherche peut etre nettement reduit en utilisant 
la methode d’apprentissage des systemes adaptatifs. L’expression 
mathematique proposee permet de ddcrire differents processus 
d’apprentissage et d’etablir des analogies entre eux. Les experiences 
relatives a l’auto-apprentissage des systemes adaptatifs ont dte con¬ 
duces a Paide de la machine dlectronique sp£ciale de l’lnstitut d’Elec- 
tronique, d’Automatique et de Tdldmecanique de l’Acad&mie des 
Sciences de Georgie. 

Zusammenfassung 

Dieser Beitrag befaBt sich mit der Frage der Verwendung selbst- 
lernender Prozesse in anpassenden Systemen mit minimaler Anfangs- 
information. Der auf dem Phanomen der Entropie beruhende mathe- 
matische Ausdruck, anhand dessen die Dynamik des Lern- und Selbst- 
lernprozesses gesucht wird, wird angegeben. Auch wird die Lern- 
geschwindigkeit angeffihrt, die einen WertmaBstab fur die Prozesse 
abgeben kann. Einige konkrete Beispiele fur das Lemen in Automaten 
und Lebewesen werden besprochen. In diesem Beitrag werden einige 
Angaben fiber die Versuchsergebnisse eines Selbstlemsystems gegeben, 
das die automatische Synthese eines Korrekturgliedes durchffihrt. 
Mit Hilfe der Lernmethoden ffir das anpassende System konnen, wie 
diese Daten zeigen, die Suchvorgange wesentlich verkfirzt werden. 
Der hier vorgeschlagene mathematische Ausdruck kann ffir die 
Beschreibung verschiedener Lernverfahren benutzt werden und macht 
Vergleiche zwischen ihnen in gewisser Beziehung moglich. Die Ver- 


suche im Zusammenhang mit dem Selbstlernen des anpassenden 
Systems wurden auf einen besonderen Elektronenrechner des Institutes 
ffir Elektronilc, Automatik und Telemechanik der Georgischen 
Akademie der Wissenschaften durchgeffihrt. 


Introduction 

At the present tune, side by side with the existing usual control 
systems, the so-called adaptive systems, which are themselves 
capable of defining the necessary structure in a given situation, 
are being worked out. As a rule the definition of a new structure 
in the adaptive system is performed with the help of the algorithm 
put in beforehand. 

Of great interest is the following step in the development of 
the adaptive system, when a device without a given algorithm 
will be capable of performing all the necessary changes according 
to the stored information. 

These systems will be different from the usual ones, working 
on the definite algorithm of search by the degree of determina¬ 
tion. The solution of this problem can be found by the applica¬ 
tion of self-learning methods. The adaptive system with self¬ 
learning properties will be generally characterized by the 
capacity of the initial information which has been put into the 
system by the designer. Devices working with the minimum 
initial information are, for example, the perceptron created by 
Rosenblatt and the homeostat first proposed by Ashby. If the 
perceptron has self-learning properties and by storage of definite 
knowledge can decrease the initial uncertainty, Ashby’s homeo¬ 
stat lacks such properties. The self-learning process can be 
fulfilled after a special storage device is added to the homeostat, 
when the system acquires the properties of self-organization. 

Dynamics of Learning Processes 

While examining the systems with the minimum initial 
information and the subsequent increase of information by self¬ 
learning processes, there arises the question of the measure of 
the initial uncertainty at which the system starts its organiza¬ 
tion. 

This uncertainty must decrease while the number of tests are 
increased. The measure of such uncertainty must be based on the 
phenomenon of entropy. But the latter needs some changes. 
Actually the entropy has its maximum value when all the events 
are equally probable. When the processes of self-learning or 
learning take place, the entropy of the system at the beginning 
of the process may have minimum or even zero value, and al¬ 
though according to Shannon the system will be fully determined, 
it may be on its lowest step of organization. This is because the 
probability of the event which the system learns has a smaller 
value than the probabilities of other events. 

As we are interested in the process of organizations of the 
system, followed by the decrease of the vagueness, the term is 
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introduced to differentiate it from Shannon’s uncertainty. Let 
us take some function V as the measure of the latter. 


_ 2 H m H S<S m 

H S>S M (1) 

where H M is the maximum value of the entropy, His the entropy 
of the system, S is the number of trials, and Sm is the trial when 
the entropy acquires its maximum value. 

As is known 

AlogiV 

i— 1 

The entropy acquires its maximum value when all the n events 
are equally probable 



then 


H m = -n —log — = Iog?j 
n ° n ° 


( 2 ) 


After each cycle a new value of V is received, according to 
the redistribution of P probabilities. Hence, there is an expres¬ 
sion 

V=cp(S) (9) 

where <p determines the learning process. 

Let us use the difference equation for the description of the 
learning process. 

Consider that V (S) is determined for the number of values 
& “h CAS, where C has the value 0,1,2,..., m. As is known, 
the difference of the first order AF will be defined as 


AF s =F s+as -F 

(10) 

the difference of the second order: 


A 2 F s =AF s+as —AF S 

(11) 

Analogically the difference of Kth order: 


A k F s = A x- 1 F s+as — a k ~ 1 v s 

(12) 


The last expression corresponds to Hartley’s information ca¬ 
pacity. Let us define it as C x . Hence, the function of vagueness 


F= 


2C 1 (n) — H S<S M 
\ H S>S M 

So the initial vagueness of the system will be equal to 


(3) 


V 0 = C(n)-H 0 (4) 

where C (ri) is the doubled Hartley’s information capacity and 
Hq is the initial entropy of the system which is determined 
according to the probability of different events at the beginning 
of the learning process. 

The maximum initial vagueness of the system is equal to 


V 0M = C(n) (5) 

and will take place when the initial entropy of the system ff 0 = 0. 
In spite of the fact that the system according to Shannon is fully 
determined, the initial vagueness of the system takes its maxi¬ 
mum value. 

Let us introduce some non-linear function X ( S) 


2(S) = 


1 

C(n) 


H 


-1 


ScS M 

S>S M 


Eqn (3) will have the final form 


( 6 ) 


V=C(n) — X(3)H(P 1 , P 2 ... P n ) ( 7 ) 

According to the above, the evaluation of the degree of vague¬ 
ness is carried out by means of the function 

V=f(P u P 2 ...P n ) = (8) 


On the other hand 


A Vs 1 \l(K-t)' Vs+n - z ^ 

Consider that the learning process, accompanied by the 
change of V, when S is increasing, is described by a non-linear 
difference equation, which is a non-linear relation of unknown 
V and its differences. The equation will have the form: 

P n A z V s + p n _ 1 A T ~ 1 V s + ... +/3 1 AV s +p o V s =0 (14) 

where /? depends on V and its differences. 

This equation may have a stable, non-stable or oscillatory 
solution and according to this the system may be considered as 
a learned one or the learning process may be unsuccessful. When 
the stable process takes place the following condition must be 
fulfilled: 

lim F=0 (15) 

S~* oo 

The difference equation may be substituted by some non¬ 
linear equation 

V x = f (F, V', V"... V ix ~ 1) ) (16) 

which may also have different solutions. 

The main characteristic feature of the learning process may 
be considered being the decrease of V vagueness which is charac¬ 
terized by V function. The reason for the latter can be found in 
the decrease of the probability of the event which is the most 
essential for the given learning process. Such event may be the 
recognition of a definite learning pattern, the determination of 
a control system, etc. 

The velocity of the learning process which conditionally will 
be called the rate (3 is the significant factor which presents the 
difference of the first order 


where i\- is the probability of the appearance of an event which 
gives the acceptable solution of the problem. 

The learning is performed by a number of cycles of trials in 
the presence of some stimuli in the case of living beings or the 
meaning of the environment in the case of automatic control. 


<5— ^(S + AS)™ ^(S) 
For a continuous process 



( 17 ) 

(18) 
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According to eqn (3) 

I (\U 

S<S M 
S>S M 

The increase of V function is determined by 


5 = 


d H 
"dS 
d H 
d S 


(19) 


Z Pi (S) P i (S) Z P i (s- 1 ) P i (S- 1) A<Sm 

AV=\ \ „ (20) 

' Z •f > i(S)l°g-f , i(S)+ Z ■Pi(S-l)l°g-f > i(S-l) A>S m 

i = 1 i = 1 

where P t (s) is the value of probability of i events in S trials and 
Pi (s~i) is the value of probability of i events in (5—1) trials. 
Hence, the rate of learning 


abilities of recognition of each recognition pattern are equal. 
The V function in this case is equal to 

V(o)= -P„logP v - Z AlogP ; (25) 

i— 1 

where P w is the probability of recognition of any one pattern 
the perceptron will learn. 

The above assumption, meaning 

P 1= P 2 =...=P V =...=P„ (26) 

cannot be characteristic for all the processes. Sometimes the 
probability of appearance of an event connected with learning 
is smaller than other probabilities. 

Each trial in the perceptron, connected with placing on the 
retina the same pattern, is followed by P v increase and Pi 

Q = l, 2.n — 1) decrease. The changing law of probabilities 

may be considered as exponential 


lim 


Z P i (S) lo g p i(S)- Z Phs-i) 1 °S p hs-d 

i= 1 _ i= 1 _ 

AS 


lim 

AS -+00 


X Pi(S-l) Pi(S- 1) X Pi (5) Pi ( S ) 
i= 1 ___ i= 1 _ 

“ — AS 


( 21 ) 


The rate of learning may be a criterion of the quantitative 
evaluation of the learning process. From this point of view it is 
convenient to analyse the learning process on the phase plane, 
the coordinates of which are the rate of learning <5 and the value 
of vagueness. In Figure 1 (b) are given several trajectories 
corresponding to the stable learning process (curve 4), to the 
non-stable process (curve 2) and to the oscillatory process 
(curve 3). 

In some cases the learning process can be finished at small 
values of the vagueness function, i.e. when 


lim V=s (22) 

S~* oo 

where 

£«C (23) 

According to eqn (7) the learning process, accompanied by 
the decrease of V vagueness, takes place because of the increase 
of one of P( t ) probabilities which corresponds to the learning 
event and the decrease of all other probabilities. The functional 
relations 

P,-/(S) (24) 

are different, and as experiments show are mainly determined 
by exponential laws. 

Later, the learning process of the adaptive system, with the 
minimum initial information at which the probability changes 
according to the exponential function, will be analysed. 


Concrete Examples of Learning Processes 

Let us examine several examples of learning processes with 
the aim of drawing analogies between them. As already men¬ 
tioned, the perceptron is a device with the minimum initial 
information. It is assumed that at the beginning of work prob¬ 


1 

P; = — e” aS 

n 

where oc is a constant. After the first trial V ( S ) according to 
eqn (7) will have the form 

= e-‘-^Ie-“log— (28) 

n n 

After the second trial: 

= e - 2 “—^e^'log— (29) 

n n 

Analogically after S trials 

^e-s^^e-^iog— (30) 

n n 

The rate of learning for the perceptron is defined by 
5 =^zle"* s log (~j- 1 + nj 

_«(n-l)e-^io g (^i—!Lzle- s j (31) 

In Figure 1 (a) is given the learning process of pattern recogni¬ 
tion. In Figure 1(b) (curve 1) is given the phase trajectory of the 
above-mentioned process. The curves are drawn for n = 20 and 
oc = 0*1. 

As seen in Figure 1(a) for 50 cycles of trials the vagueness of 
the system in the learning process decreases from V( 0 ) = 4*32 
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Figure 1 

bit to V — 0*12 bit, i.e. by 36 times. By further increase of the 
number of cycles V can be decreased to the arbitrary small value. 

Now consider the learning process for reproduction of words. 
The process is based on the data obtained in the experiments 
made by Bruner and Zimmerman 1 . The experiment consists 
in the following; a list of 32 mono-syllable words is read to the 
person being taught. The person then writes down those words 
which he has remembered. Then, to avoid correlation, the order 
of words is changed and the process is repeated. 



Figure 2 


Of interest are the experiments effected by Solomon and 
Win. In one experiment a dog is taught to avoid pain which is 
the result of a current. According to the data given, the cor¬ 
responding learning process calculated by eqn (7) is shown in 
Figure 2(a) (curve 2). 

In Figure 2(b) is given the corresponding phase trajectory 
(curve 2) for which the equilibrium point is not at the origin of 
the coordinates. This means that after the learning process is 
finished, vagueness equal to 

li raV(S) = e (32) 

is left. s ~* 00 

The learning process of a hungry rat which is placed into a 
T-shaped labyrinth is similar. In Figure 2 (a) and 2 (b) curve 3 
shows the character of changes of V vagueness with increase of 
the number of cycles of trials and the phase trajectory of 
the process. In this case the vagueness decreases by 12*5 times 
in 30 trials. The phase trajectory is compressed to the origins 
of the coordinates and 

lim V(S)=0 (33) 

S'-* oo 


The Learning Process of Adaptive System Fulfilled by the Ex¬ 
clusive Method 

Consider the self-learning process of the adaptive system 
working on minimum initial information. As is known, such a 
system defines some subset M by random search. The elements 
of the subset are the points corresponding, from some point 
of view, to the position of any vector Y, which characterizes 
the state of the system. If A is the set of all possible states, then 
Ma N. According to the above, when Y is the domain defined 
by M subset, the search process and the learning process of the 
adaptive system must be finished. First, the direction of Y is 
defined according to the law of random numbers, because of 
the lack of some information. Then the evaluation of the given 
domain takes place and if the latter does not correspond to 
requirements, the coordinates of the vector domain will be 
stored by a special memory device. In the following cycles 
of trials Y cannot return to the already tested position cor¬ 
responding to the set. Thus the exclusion of one element from A 
set takes place. Then the system acts analogically. Assume that 
A and ]\d are restricted final sets. By m and n designate the power 
of final sets of M and A correspondingly. As P probability of 
any Y position is equal to P = 1 /«, P ip probability of Y position 
defined by M set will be equal to 


In Figure 2 (a) is given a curve of learning process (curve 1) 
according to eqn (7). P(S) is determined according to the ex¬ 
perimental data. In this case there are two events: (1) the 
remembering of the whole table, and (2) the case of non¬ 
remembering. V is determined by the probabilities P and 1_ P. 

In Figure 2 (b) curve 1 corresponds to the phase trajectory of 
the process. As seen in Figure 2 (a) the experiment was finished, 
when ¥% (S) = 0*4 bit. The initial vagueness is equal to V M (0) 
= IT bit. In this case the learning process cannot be considered 
as fulfilled, because the relation V M (S)/V K (S) is sufficiently 
small. But the extrapolation of the curve allows the conclusion 
that when 5 — 50, V K (S) = 0*02 bit. In this case the vagueness 
decreases by 55 times. 


m 


P V = . I p >=~ (34) 

i — n — m 


The value of probability in the following cycles will be 
1 „ m 


P = - 


n — S 


77—5 


(35) 


In the considered adaptive system the direction of the vector in 
the presence of storage is defined by the amplifier gain in four 
different channels each of which can obtain seven discrete 
values. The total number of sets of the system is equal to 
n = 7 4 = 2,401. In the adaptive system the process of self¬ 
learning begins at the smaller values of information capacity, 
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as m > 1 and P m > P r The information capacity coinciding 
with the value of the initial vagueness of the system is defined as 


K (or 


^4log P A - E PjlogPj 


J= i 


(36) 


The vagueness of the adaptive system after S cycles will be equal 
to n — S 


■/ m ^ m V" l , l 
h, ^ =-- log -— ? - E -—o log 


<o>- n _ S "*n-S n-S 
After some transformations 


n — S 


(37) 


Us) =- "F log m + log (n - S) 

n o 

The rate of learning 


S 


dV 
d S 


m 


log m — 


(38) 

(39) 


Eqn (39) shows that the rate of learning increases in absolute 
value with the increase of number of S trials. Since 0 < S < 
n — m, the rate of learning reaches its maximum at the end of 
the learning process when 

S = n — m 


According to eqns (38) and (39), V, the vagueness of the system, 
and (3, the rate of learning, depend also on the power of sets M 
and N. 

Let us consider now some concrete examples of work of the 
adaptive system when the adaptive systems automatically define 
the structure and the value of parameters of correcting devices. 
The domain of the adaptive system is evaluated by the well- 
known integral criterion. 

The learning process consists in the following: the model 
of an object, the equation of which is known, is connected to the 
adaptive system, which changes its structure and parameters 
till satisfactory work of the whole system is accepted. The 
storage device docs not allow the repetition of the former state. 
Such a process, which can conventionally be called the learning 
process, by the exclusion method does not essentially differ 
from the analysed learning process of a dog and a rat. 

The results of the experiments of the work of the adaptive 
systems by the definition of the required structure of the cor¬ 
rective devices arc given below. Here is an automatically con¬ 
trolled plant which is described by the equations: 

x 1 =30 z —x 3 —x 4 

0-025 x 2 + 0* 125 x' 2 + x 2 = 4 x , 

x' 3 -hx 3 = 1-5 x 2 

The equation of the corrective device in the presence of which 
l[oc t xi + a 2 (x' 3 ) 2 ]dt^A 
must be determined. 

The adaptive system without the use of a memory device 
could not define in 8 h the corresponding structure. The search 
was fulfilled using self-learning methods and after 1,503 cycles 
of trials, taking 5 h 12 min, the structure of corrective device 
described by the equation: 

0 * 01845 x' 4 4- x 4 = 0-0063 x 2 4-0-282 x 2 
was defined. 


Repeating the search process, and having experience in the 
learning process, the adaptive system defined the new domain 
of the system, satisfying the given conditions, in 14 min after 
23 cycles of trials. In Figure 3 (curve 1) is given the learning 
process computed according to eqn (37). In Figure 4 (curve 1) 
is given the phase trajectory of the learning process, defined 
according to eqn (39). In the given example only 12 domains 
from 240 satisfy the given conditions, i.e. m = 12. For the 
second example let us take the process of definition of the 
structure of parameters of a corrective device intended for some 
plant, described by 

0-0005 xi" + 0-051 x'/ +1 -06 xi 4* 23 Xi = 22 (z - x 2 ) 




In Figure 3 (curve 2) is given the corresponding self-learning 
process. In Figure 4, curve 2 shows the corresponding phase 
trajectory. The equation of the corrective device, found as a 
result of self-learning, has the form 

0-0001 x 2 + 0-01 x 2 4* x 2 — 0-001 x' x 

As in this case m = 32, the self-learning process is fulfilled a 
little quicker. Curve 3 in Figure 3 corresponds to the self¬ 
learning process of the system, when the initial vagueness is 
defined by m = 72. The initial equation of the plant is 

28-06 x[ 4- x x = 8-98 (z - x 2 - x 3 ) x' 2 =. x 


457 






V. K. CHICHINADZE 


The equation of the corrective device found by the adaptive 
system as a result of the learning process has the form 

0*666 x 2 + x 2 + x 3 = 0*666 x[ + 0*666 x^ 

The phase trajectory of the process is given in Figure 4 (curve 3). 

Let us analyse the last example, which corresponds to the 
relatively small value of the initial vagueness, when m — 194. 
The curve of the learning process is given in Figure 3 (curve 4). 
The phase trajectory is given in Figure 4 (curve 4). The equations 
of the plant have the form 

0*15x' +x 1 =2 (z — x 4 ) 

0*1 x 2 + x 2 = 2 ;c 1 

X 3 + x 3 = 50 (x 2 — x 5 ) 

1*5 x' 4 + x 4 — x 3 

The equation found by the adaptive system has the form 
X 5 +1*3 *5 + 0*36 x 5 = 0*1 X 3 + 0*3 x 3 

The phase trajectory of the learning process of the adaptive 
system given in Figure 4 and also eqns (38) and (39) shows 
that the <5 = y (V) function in V — 0 point has a discontinuity 
of the first order when in the analysed examples the rate of 
learning was smoothly decreasing. The reason is the fact that 
learning by the exclusion method with the accepted assumptions 
is accompanied by no essential decrease of the vagueness at the 
beginning of the learning process and increase of velocity 
with the storage of information. The above follows the relation 
P— mjn — S. The opposite applies using other learning methods. 
At the beginning of the analysed process the decrease of rate 
of vagueness is relatively higher than the exponential velocity 
decrease. The self-learning process is based on the assumption 
that Y in the searching process does not enter the domain which 
is determined by M subset. Such a restriction is a strong one. 
If the process can be considered without such a restriction, then 
for the determination of the law of changing of the probabilities 
for Y entering this domain is determined by the hypergeometric 
law. The probability of Y passing some points m K (0 < m K < m) 
will be determined by 



Hence for each S value the probability W l9 fo Y entering the 
M domain can be determined and 

m 

I W K (41) 

K-l 

According to eqns (7), (40) and (41), giving different values for 
S can be determined by the function 

W=f(S) (42) 

In Figure 5 {a) are given the curves of the self-learning 
process by the exclusive methods computed according to 
eqn (7) for the cases when n = 52, m = 4 (curve 1) and n = 10, 
m = 2 (curve 2). 

In Figure 5(b) are given the corresponding phase trajectories. 
The data received as a result of experiments shows that the 
exclusive method can be used for cases when the initial vague¬ 
ness has a sufficiently large value. The advantage of the exclusive 
method is that the work can be fullfilled without any search 
algorithm. 

The experiments on self-learning were effected on a specialized 
computer which was designed at the Institute of Electronics, 
Automatics and Telemechanics of the Academy of Sciences of 
Georgia. 
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On the Searching of Extrema of Functions 
in Automatic Control Systems 

A. A. VORONOV and M. B. IGNATJEV 


Summary 

A peculiar approach to the problem of searching extrema of functions 
of many variables on the intersection of multidimensional surfaces is 
considered. The differential equations the solutions of which are set on 
these multidimensional surfaces (manifolds), or on their intersections, 
are given. Ip the structure of these equations the arbitrary functional 
coefficients ‘w s ’, which determine the trajectory on the surfaces, are 
revealed. In case of reproducing the manifold F (x l9 x 2 ,. -. x n ) = 0, the 
number of arbitrary coefficients is equal to Cg; in case of the trajectory 
lying on the intersection of 6 ra’ manifolds Fj(x l9 x 2 , ... x n ) — 0, 
i = 1, 2, ... m, the number of arbitrary coefficients is equal to CJ? +1 . 

Having certain concrete arbitrary coefficients c u$ one can make 
devices performing the continuous searching of extrema of functions 
of many variables, the extremum point being the steady-state point for 
the structures considered. By choosing different coefficients it is 
possible to obtain the movement towards the extremum point along 
various trajectories (in particular along the geodetic curve). For 
example the differential equations, the trajectories of which are located 
on the surfaces F(x, y, z) = 0 are: 

dx bF 5 F d y 5 F , 5 F dz bF 5 F 

„ Ux +« 3&z , df ~ u 2 dx »» dy 

and the movement towards the maximum of z takes place when 

« 2 = , u 3 =-al^ 

The considered structures may be used for optimization of a 
composite parameter for complex systems, for solving the problem of 
construction of robot’s motion, and for construction of ultra-stability 
systems. 


Sommaire 


On examine une solution particuliere pour resoudre le probleme de 
recherche d’extrema de fonctions de variables multiples aux intersec¬ 
tions des surfaces multidimensionnelles. On donne les equations diffe- 
rentielles dont les solutions se trouvent sur ces surfaces multidimen¬ 
sionnelles ou k leurs intersections. On localise les coefficients arbi¬ 
trages fonctionnels «u s » qui determinent la trajectoire sur les sur¬ 
faces. Dans le cas de la polyvalence F(x lf x 2 ,... x n ) = 0, le nombre 
de coefficients arbitrages est 6gal a Cg. Dans le cas ou la trajectoire 
se trouve a l’intersection de «m» surfaces F$(x l9 x 2 ,... x n ) = 0, 
/ = 1, 2, ... m 9 ce nombre est egal a C% +1 . 

Avec certaines valeurs concretes des coefficients arbitraires u s , on 
peut construire des dispositifs pouvant effectuer la recherche continue 
d’extrema de fonctions de variables multiples, le point extremal repre- 
sentant l’etat stationnaire des structures considerees. En choisissant 
convenablement ces coefficients, on peut realiser le mouvement vers 
le point extremal selon des trajectoires varies (notamment le long de 
la g6odesique). Par exemple, les equations differentielles dont les sur¬ 
faces Fix, y, z) = 0, sont: 


dx ' 5 F bF 

dt ~ Ul dy U2 bz 5 


dy bF . bF dz bF bF 

~dt “ ~ Ul ~bx ^ Uz bz * d t ~ U2 bx U 3 by 


et 


le mouvement vers le maximum de z a lieu jquand: 


llo — 



Ui = - a\ 


chF 

by 


Le processus considere peut etre utilise pour Toptimalisation d’un 
parametre compose dans les systemes complexes, pour la construction 
du mouvement d’un robot et de systemes ultra-stables. 

Zusammenfassung 

Ein spezieller Zugang zu dem Problem der Suche nach den Extrem- 
werten von Funktionen mit mehreren Veranderlichen an den Schnitt- 
stellen der mehrdimensionalen Flachen wird betrachtet. Die Differen- 
tialgleichungen, deren Losungen auf diesen mehrdimensionalen 
Flachen (Mannigfaltigkeit) oder an ihren Schnittstellen festliegen, sind 
angegeben. Der Aufbau der Gleichungen enthalt die willkiirlichen 
Funktionen „u s “, welche die Trajektorie auf den Flachen bestimmen. 
Im Falle der Darstellung der Mannigfaltigkeit F (x l9 x 2i ... x n ) = 0 ist 
die Anzahl der willkiirlichen Funktionen gleich Cg; liegt jedoch die 
Trajektorie an den Schnittstellen von „m“ Mannigfaltigkeiten Fj(x lt 
x 2i ... x n ) = 0 O' = 1, 2, ... m ), so ist die Anzahl der willkiirlichen 
Funktionen CJf +1 . 

Hat man bestimmte konkrete willkiirliche Funktionen u Si so lafit 
sich eine Einrichtung finden, die kontinuierlich nach den Extrem- 
werten der Funktionen mehrerer Variabler sucht; der Extremwert 
stellt den Endpunkt (eingeschwungener Zustand) der betrachteten 
Struktur dar. Durch die Wahl von verschiedenen Funktionen erreicht 
man, daB die Bewegungen zum Extremwert entlang verschieden- 
artiger Trajektorien (besonders entlang der geodatischen Linie) ver- 
laufen. So sind zum Beispiel die Differentialgleichungen, deren 
Trajektorien auf der Flache F(x , y, z) — 0 liegen, 

dx bF bF dy bF bF dz bF bF 

d 7 = Ul 6^ _ " 2 d? ~dt = ~ Ul te + “ 3 &7’ 17 ~ “ 3 67 

Die Bewegung auf das Maximum von z hin findet statt, wenn 

9 dF 2 bF . + 

u 2 = u z =- ~ a\— 1 st. 

Die betrachtete Struktur kann zur Optimierung eines komplexen 
Systems, zur Losung des Konstruktionsproblems einer Roboter- 
bewegung und zur Erstellung eines ultrastabilen Systems dienen. 


This paper considers a distinctive approach to the problem of syn¬ 
thesis of local systems for automatic search of extrema of func¬ 
tions of many variables. The principle involved in the construction 
of systems which react upon the partial derivatives of the sought 
function by the coordinates of the controlling devices is not new. 

In the search of the extremum of the function of a single 
variable the problem is sufficiently defined; however, when the 
function depends upon several variables, the definiteness is lost 
and the solution of the problem becomes multi-valued. The 
function of a single variable y ~ fix) is represented by a plane 
curve, and if at a certain point x x one determines dy/dx, then 
it is necessary to vary x in order to approach the required 
extremum. A function of two variables may be represented by 
a surface y — f (x l9 x 2 ). In this case, the path followed in passing 
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from a given point to the point of extremum, while remaining 
on the surface, is not a singular one, but one of infinite multitude. 

In 1959 Krasovski considered systems which searched the 
path to the extremum by the gradient method 7 . He also showed 
that depending upon the form of the surface;; = f(x l9 x 2 , ..., x n ) 
the gradient method may be varied by making the shift of the 
controlling device x { dependent on the derivative = d//dx* 
and also upon the derivatives of the function / with respect to 
other coordinates. 

Approximately at the same time the Electro-Mechanical 
Institute of Leningrad considered the problem of simulating 
functions of many variables by means of digital differential 
analysers. This problem arose in connection with the construc¬ 
tion of systems of programme control of metal cutting machines, 
first for simulating plane curves and then for curves lying on 
a given surface. The method which was utilized in this instance 
made it possible to indicate the general methods of synthesis 
with DDA (digital differential analyser) intended for simulating 
various forms of multidimensional surfaces, and also to indicate 
the quite general method of constructing systems for searching 
of extrema of functions of many variables, based on the principle 
of partial derivatives measurement. The gradient method is 
obtained in this instance as a special case. This method also 
permits the searching of extrema, taking into account the bound¬ 
aries at the intersection of multidimensional surfaces. 

Before proceeding to the treatment of this method, it is 
necessary to consider the question of the structure of differential 
equations whose solution lies at the given intersection of 
multidimensional surfaces 4 ’ 5 . 

The Structure of Differential Equations whose Solution Lies at the 
Intersection of Multidimensional Surfaces 

The problem of finding differential equations whose solution 
is a given function is not a single-valued problem and its 
rational solution depends upon the means which are used for 
composing the sought system of equations, and upon which 
properties of the functions are utilized in the solution. 

Indeed, in mathematical analysis there are given proofs of 
theorems on the singularity of solution of differential equations 
under given conditions; however, it is obvious that the converse 
problem has a multitude of solutions, that is, it is possible to 
find a multitude of differential equations whose singular solution 
will be the given function. 

At the present time there exist two approaches for solving 
differential equations. The first approach permits the construc¬ 
tion of an equation by introducing a parameter, and in the 
following this approach is called the parametric method of 
synthesis. The second approach of developing the method of 
synthesis is one which converts an equation into an identity, 
this equation being obtained by differentiating the output func¬ 
tion with respect to the parameter. 

Let there be a function 

F(x u x 2 ,...,x n ) = 0 ( 1 ) 

and an argument <p. In the parametric method of synthesis one 
finds first of all the parametric equations 

= z = l,2,...,ft (2) 

which satisfy (1), and from these equations differential equations 
are found whose solution will be given by functions (2). In this 


case it is possible to find the differential equations by the method 
of K (D) transform proposed by Kulebakin. 

By the second method 4 ’ 5 the parametric expression (2) is 
not sought, but the differential equations whose solutions 
satisfy (1) are immediately determined. As numerous observa¬ 
tions indicate, the structures synthesized by the second method 
are considerably simpler than the structures synthesized by the 
first one. 

The basis of this method of analytical construction of a differ¬ 
ential analyser is the following lemma: given the function (1) 
of ft variables which has a derivative in the given range of the 
variables, then in order that the solution of differential equations 

dX; 

* = ...,/2 ( 3 ) 

under initial conditions satisfying (1) may transform eqn (1) into 
an identity, it is necessary and sufficient that the eqns (3) trans¬ 
form into an identity eqn (4): 


” 0F cbq 
dx t dcp 


(4) 


This lemma is then utilized for searching functions j\\ 
they are sought so that they may transform eqn (4) into an 
identity. This problem has a multitude of solutions, and this is 
what determines the fact that the problem of synthesis is not 
a single-valued one. No matter how we may determine f h 
they will in all cases be some functions of partial derivatives 
bF/bx^ In the operation the functions are sought out as 
linear functions of partial derivatives under the assumption that 
this is the simplest case. 

As is shown 4 ’ 5 the differential eqns (3) whose solutions satisfy 
(1) include arbitrary functions U s whose number is $^C n \ and 
the matrix of these arbitrary functions is symmetrical with respect 
to the diagonal with zeros along the principal diagonal. For 
instance, the structure of differential equations whose trajectories 
are disposed on surface F (. x , y, z) = 0 is determined by equations 


dx 

dF 

0 F 

dcp 

Ul dj~ U2 

dz 

dy__ 

dF 

dF 

dcp 

Ul d^ +U3 

dz 

dz 

dF 

dF 

dcp 

U2 ~-u 3 

dy 


where u l9 u 2 , % are arbitrary functions which determine the 
trajectory on the surface once they are given. They may be any 
functions as long as they satisfy Lipschitz conditions for right- 
hand sides of differential equations. 

In simulating the trajectory at the intersection of surfaces 


Fj( x i> x 2 ,x n ) — 0 y=l,2, ...,m (6) 

men 


the number of arbitrary functions in the structure of differen¬ 
tial equations is determined as 


S = C +1 (7) 

and it is possible to determine the disposition of these arbitrary 
functions in the structure of the equations. 
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As an illustration of these methods of synthesis differential 
equations will be found whose solutions are disposed on surfaces 

F J (x 1 ,x 2 ,x 3 ,x 4 )=0, j= 1,2 (8) 

At first the arbitrary functions are designated, the number of 
which in this case is C 4 3 = 4, 

1/ 1 = C 12 3j w 2 = ^1245 W 3 =: ^134 j U A ~ ^234 


The coefficient in which the subscript of the term C contains 
unity are disposed in the first line; the coefficients in which this 
subscript contains the number two, are situated in the second 
line, etc., that is, 

dxit i i 

U 4 D 2 3 “b w 2^24“b u s^34 

dx 

-~^= —u 1 Di Z —u 2 D1 4 ' + u 4 .D1 4 . ^ 

dx 

u iD\ 2 — u 3 D\ A . — u A P\ Ar 

lf 2^12 + W 3^13 + W 4^23 

where the letter D designates the sum of the products of partial 
derivatives of the function (8) with respect to variables whose 
subscripts are present in the subscripts of the term D. 


__ 8 Ft QF 2 dF t 8 F 2 
12 8 xi 3 x 2 3 x 2 dx x * 


etc. 


The superscript of the term D denotes the line. 

The signs before the terms in eqns (9), as can be shown, are 
determined by the following rule: consider the order of the 
superscript and subscript of the symbol D, for instance that 
normally indicated by a pointer, and if there is an odd number 
of violations of the normal order, a minus sign is used before 
this term, and in other cases a plus sign is used. That is, for 
terms of the top line one has orders 123, 124, 134 in which 
there are no violations, and these are accompanied by a plus 
sign; in the second line there are 213 with one violation (2 being 
greater than 1), and a minus sign is used; 214 with a minus sign, 
234—no violations—a plus sign; in the third line, 312—two 
violations—plus sign; 314—one violation—minus sign; 324— 
one violation—minus sign; in the fourth line, 412—two viola¬ 
tions—plus sign, and 413 and 423 also have plus signs. 

In an analogous manner one determines the structure of 
differential equations and the signs and disposition of arbitrary 
functions in the latter by simulating trajectories at any number 
of intersecting surfaces with any number of variables. 

It is of interest to note the presence of a maximum with 
respect to the number of arbitrary functions in the structure of 
differential equations for systems with a number of variables 
greater than six. The number of arbitrary functions for n < 6 
decreases as the number of intersecting surfaces increases. 
For n >6 the number of arbitrary functions for an increasing m 
at first increases, and only after having attained a maximum for 
m = (n — 2) with even values of n, and for m = (n — 2 ± 1/2) 
for odd values of n , does it begin to decrease. 

The arbitrary functions u s in the structure of differential 
equations may be utilized as means of control in specifying 
prescribed motions on multidimensional surfaces, and as means 


of self-tuning of an automatic control system. Formula (7) 
relates the number of dimensions of the control space to the 
number of degrees of freedom of an automatic control system, 
and to the number of constraints imposed upon the system, 
while the presence of a maximum in the number of arbitrary 
controlling functions indicates an optimal structure as regards 
self-tuning of a system for n > 6. 


Determination of Extrema of Functions 


The problems of searching out the extrema of functions is 
one of the most widely encountered ones. There exist different 
methods of finding extrema in the presence of known partial 
derivatives, and different methods of automatic determination 
of these partial derivatives. However, at the present time, the 
methods of searching the extrema of functions in the presence 
of constraints placed upon the variables are not sufficiently 
well developed, and none of the existing methods assures that 
the motion to the extremum will proceed along a geodesic, or 
the shortest line. 

In order to find the extrema one may utilize arbitrary 
coefficients in the structure of differential equations. Indeed, in 
order to assure the motion to an extremum-maximum with 
respect to coordinate x i9 it is sufficient to prescribe such a motion 
that the coordinate x z - increases all the time, and this may be 
achieved by specifying the coefficients u s in a proper manner. 
For instance, in order to attain a maximum with respect to z on 
the surface F(x , y 9 z) = 0 it is sufficient to assume in the system 
of eqns (5): 


In this case 


u 2 — a\ 


dF 

8x’ 


U 3 — ““ a 2 


8 F 
8 y 


dx _ a F_ 2 dFdF 
d cp Ul 8 y a± 8x dz 


dy___ 8 F_ 2 8F8F 
dcp Ul dx 0,2 dy 8z 



( 10 ) 


where dz/d<p will be a positive definite form of a constant sign 
for all real values of x, y, z, which assures the stability of the 
process of finding the extremum in accordance with Liapunov 6 . 
At the point of the maximum with respect to z, the velocities 
with respect to all coordinates become zero. For a system of 
eqns (10) the point of maximum with respect to z proves to be a 
point of stable equilibrium. In the motion toward the extremum- 
minimum 


u 2 = 




dx’ 


l Qy 


and dzjdcp will be a negative definite form. 

In an analogous manner we determine the coefficients u s for 
a specified motion toward the extremum for surfaces with a 
large number of dimensions as well. 

The synthesized structures may be utilized for searching out 
extrema of functions with any number of variables for individual 
surfaces as well as for cases in which constraints are taken into 
account, that is, for intersecting surfaces. For instance, in 
searching the maximum with respect to coordinate (x 4 ) at 
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the intersection of surfaces (8) for a specified motion toward 
this extremum it is possible in the system of eqns (9) to let 

i/ 2 = j D 12 , u$ = D 13 , U 4 — D 23 

and dxjdcp will be a positive definite form, and this fact assures 
stability of the process of searching the extremum according to 
Liapunov. 

For a motion toward the extremum prescribed in this manner 
there remain free arbitrary functions in the synthesized struc¬ 
tures, the number of which functions is equal to: 

c _ S^m— 1 y'-'fWJ 

These free arbitrary functions may be utilized for correcting 
the trajectory during the time of the motion toward the extremum. 
In the example considered above there remains a free arbitrary 
function ii± in the system of eqns (10). The free arbitrary func¬ 
tions may be utilized for prescribing the motion toward the 
extremum along a geodesic curve or one which is close to it. 

It should be noted that all stationary points for the obtained 
differential equations will be points of equilibrium, but only 
points of the extrema will be points of stable equilibrium, while 
the saddle points will be points of unstable equilibrium. 

If the number of intersecting manifolds is m = n - 1, then 
they determine a line in the ^-dimensional space. In this case 
the problem is reduced to searching out the extremum in a 
one-dimensional manifold. It may be assumed that dcp = codr, 
where r is the time and co is an arbitrary function which satisfies 
the Lipschitz condition, and 

dx £ x 

—CO Si (*^15^25 * * * 5 i — 1, 2, . .. 5 Cl 

For prescribing the motion toward the extremum in this case 
it is only necessary to specify the direction of the motion along 
the line. For example, in the motion toward the maximum with 
respect to x n it is sufficient to assume that co = g n (x x , x 2 ,.. x n ), 
and then 

dx- 

^ = (*^15 ^2? •••? x n) * ^2? •••? •X'n) 


/ 

dr 1X15 


• •••>*!») 


There follows a comparison of the described method of searching 
out the extrema and the gradient method. As shown by 
Krasovski 7 , the gradient method assures stability, according 
to Liapunov, in the computing process of searching the extre¬ 
mum. This constitutes the similarity between them. But the 
gradient method assures the displacement toward the extre¬ 
mum only along some special trajectory, while the proposed 
method permits the variation of trajectory of motion toward 
the extremum. 

Indeed, in the system of eqns (10) there remained one free 
arbitrary coefficient u which may be specified by a different 
method and which supplements the definition of trajectory for 
the motion toward the extremum. An analogous situation exists 
also in searching the extremum for other manifolds or their 
intersections, except for those which are one-dimensional. The 
gradient method constitutes a special case of the considered 
method of searching the extrema, when all the remaining 
arbitrary coefficients are set equal to zero; for instance, for the 
system of eqns (10), when u x = 0. 

The remaining arbitrary coefficients may be prescribed in 


such a manner as to assure the motion toward the extremum 
along a trajectory which is optimal in some sense, including in 
this number a geodesic trajectory. 

Figure 1 shows a block diagram of a system which searches 
out an extremum at the intersection of surfaces. The controlling 
signals produced by an analogue programming device (PD) are 
supplied to several simultaneously optimized plants 0 l5 0 2 , 
..., 0 m . In these plants the current values of partial deriva¬ 
tives which are supplied to the programming device are deter¬ 
mined in some manner. The programming device constitutes 
a differential analyser (in particular, an electronic analogue 
installation) whose structure was described in the preceding 
paragraph. The setter of trajectories (ST) carries out such pre¬ 
scriptions of the arbitrary coefficients which remain free after 
the prescription of motion toward the extremum in order to 
assure the displacement toward it along some desired trajectory. 



Figure 1 Trajectory setter 


it me equations F, t (x lf x 2 , ..., x n ) = 0, j = 1, 2,.. .,m are 
known, then 0 1? 0 2 ,..., 0 W are simply functional transforms. If 
only a part of these equations is known, this means that a part 
of Ql, 0 2 ,..0 m are functional transforms (computer assemblies), 
while the other part are the plants. 

In Figure 2 is shown the block diagram of a system which 
utilizes the method of searching the extremum described above. 
As an example consider the case of searching the maximum on 
a surface F (x l9 x 2 , x 3 , x 4 ) = 0 with respect to coordinate x 4 . 
The structure of the analogue device in this case is defined by 
equations 


dx x _ dF 
dcp Ui dx 2 


dF 

1 dxo 


dFQF 
dx x dx y 


dx,_ dF_ dF__dF dF 
dcp 1 dx^n^ 0x 3 0x 2 dx y 


0x 3 dF dF dF dF 

dcp Ul dx x U4, dx 2 0x 3 0x^ 

d x y ( 0F\ 2 /0FV (dF \ 2 
dcp \dx 1 / \dx 2 J \0x 3 y 
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In this instance we assume that bF/bx^ — — 1. The current 
values of partial derivatives may be determined by the method 
of synchronous detection. The considered system for u x = « 2 = 
u z = 0 is transformed into a scheme of extremal system cited 
by Krasovski 7 and it differs from this scheme by the introduc¬ 
tion of cross-links supplied to the input of the integrators. At 
the same time, the coefficients u l9 u 2 , u 3 may be either constant 
magnitudes or functions of coordinates x h and be controlled by 
some index of the quality of operation of the system. 

In specifying the motion along a geodesic curve in eqn (10), 
the free coefficient u lf for instance, may be determined from the 
condition that for a geodesic curve the main normal to the 
curve coincides with the normal to the surface, and at the same 
time u x is determined as a complex function of coordinates. 

If we search an extremum with respect to coordinate y 
on the surface F (x, y, t) = 0, where t is the time, then the 
structure of the analyser which specifies the motion toward the 
extremum will be defined by equations 

dx__oF0F_ 0F 

d t Qx dy Uz dt 

dyfQF Y (W V 

dt \px) \0f / 

Uz dx dt dy 

As can be seen, by virtue of the last equation of this system 
of equations, the number of free arbitrary coefficients decreases. 

It is possible to determine such constant coefficients u s which 
assure the motion toward the extremum, perhaps not along the 
geodesic curve but at least along a path which is shorter than 
the trajectories followed during the motion toward the extremum 
by the gradient method, that is, when the free arbitrary coefficients 
are equal to zero. During the motion along a geodesic curve 
these coefficients in the general case will be complex functions. 
For constant free arbitrary coefficients, the technical realization 
of the proposed method is considerably simplified. 


On the Search of Extrema of Functions in Automatic Control 
Systems 

The operation involved in searching out extrema at the 
present time is automated to a large extent and may be used as 
a basis of construction of various automatic control systems. 
In the case of a limited range of change of variables the extremum 
may be sought taking into account the constraint. 

X xf = R 2 

The method described above permits this approach. 
Frequently in controlling chemical production of great com¬ 
plexity the problem of optimization of the free index of the 
quality of the process arises; for instance, if there is a plant 
with a characteristic F(x, y } z) = 0, and it is required to determine 
such values of x, y, z which would provide an extremum to the 
free index z — / 3 x -f- m 3 y + n z z where / 3 , m z , n z are constant 
quantities, then this problem may also be solved on the basis 
of the method considered above. 

Rewriting these equations using other designations, one has 


Fi (x u x 2 ,x 3 )=0 


F 2 = l 3 x 1 +m 3 x 2 + n 3 x 3 —x 4 = 0 


The structure of differential equations whose solution lies at 
the intersection of these surfaces is determined as (9), where 


D i 


1 Sx x 


6 F\ 
dx 2 


D 13 = n 


d h 


0J\ 

dx 3 


n dF * „ 0Fi 0F t 

D ^-^ D ^ W 2 ~ m ^ 3 


n -- d J± D 

D2y_ 0x 2 ’ 3y_ 


8Fi 

dx 3 


The partial derivatives of the characteristic of the plant may 
be determined by some automatic method 1 ’ 2 . 

The problem considered above may be formulated as a 
problem of searching an extremum in a given direction, which 
is characterized by coefficients / 3 , m z , n z . At the present time 
an effort is being made to utilize the operation of searching 
extrema for solving the problem of constructing the motions 10 . 
The problem of constructing the motions based on synergy 
levels 11 ’ 12 may be formulated for the given kinematic scheme in 
terms of the intersections of the manifolds, and the motions 
themselves may be regarded as a solution of the problem of 
searching an extremum in a given direction. 

In conclusion, consider the problem of possibilities of a 
global search. The finding of an extremal extremum requires 
a more thorough study of the investigated functions, and at the 
present time various strategies for solving this problem 1 ’ 10 ’ 13 
have been proposed. One can propose yet another strategy for 
solving this problem as follows. Suppose that it is necessary 
to find the maximal maximum. Having investigated the function 
and having found several maxima, it is possible to pass a 
surface through them and the maximum of this surface will be 
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m ^ as m t ^ le zone of gravity of the sought maximum of the 
the n > a * ^ a PProximated surface will have several maxima, 
1 ma Y ke smoothed in the same manner by finding the 
s tpfH a PP r oximated surface, etc. The number of approximated 
- i aces will be determined by the complexity of the investigated 
i unction. 
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DISCUSSION 


R. Kulikowski, Polish Academy of Sciences, Warsaw, Poland 

l he random behaviour of the plant characteristics is a distinctive 
feature of the problems which are connected with extrema searching 
in automatic control systems. Therefore it is very important to mini¬ 
mize the searching time of the risk function associated with the given 
performance measure and plant characteristics. The performance meas¬ 
ure is usually a functional of the time-dependent control signals rather 
than a function of the static input values. Consequently, the generalized 
gradient which determines the changes of the control signals is a time- 
dependent operator. In order to find the extremum by the steepest 
descent, or other similar method, we have to identify the generalized 
gradient at every step of iterations. All these factors complicate the 
problem under consideration. However, some important questions 
connected with the minimization of the risk function were already 
considered by A. A. Feldbaum in his papers on the so-called dual 
control. The difficulties connected with the identification of the general¬ 
ized gradient are considered in my paper presented at this congress. 

In view of these difficulties it would be interesting to know what 
kind of practical problems can be treated by the method described in 
the very interesting paper of A. A. Voronov and M. B. Ignatjev, and 


how the system operates when the memory and random changes of 
the plant characteristics cannot be ignored ? 

A. A. Voronov, in reply 

Two important problems not considered in the report are: (1) dynami¬ 
cal properties of the controlled system and (2) the influence of random 
noise on the searching of extremum. 

Dynamics, in general, may be taken into account by the addition 
of the dynamic equation to the ones considered; this equation may be 
treated as one of the equations of surfaces (or manifolds). Thus the 
method may be applied in this case also. 

The influence of random noise demands a special investigation. 
This question is very important, yet the subject of the report was not 
the dynamics of searching but only the principle of the device which 
performs the moving to the extremum of the set function. 

The minimization of searching time is a very important problem. 
We may minimize the time if we choose the correct method of search¬ 
ing. The correct method, however, depends on the form of function F 
{xi,xv). In some cases the device considered may be useful for 
discovering the correct method of searching. 
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Dominant Operators Approach to the Theory of 
Adaptive Control Systems 

A. STRASZAK 


Summary 

This paper considers an approach to the synthesis of an adaptive control 
system that provides for rapid adaptations. It leads to a model adaptive 
control system, but not to a model comparison system. A model is 
used as a dominant operator, and then the dynamics of process may 
be neglected. Time-domain synthesis of dominant operator is intro¬ 
duced. In the time-domain synthesis, performance index approach and 
model comparison approach may be transferred to the desired form of 
differential equation of error signal. Advantages of this type of adaptive 
control system are studied. An example and some experimental results 
are given. 

Sommaire 

Ce papier presente une methode pour la synthese d’un systeme de com- 
mande adaptative permettant des adaptatations rapides. Le systeme 
comprend un modele qui n’est pas utilise comme comparateur, mais 
comme ‘operateur dominant’. De ce fait, la dynamique du processus 
commandd peut etre ndligde. On introduit la synthese de l’operateur 
dominant dans le domaine du temps. Dans la synthese, effectuee dans 
le domaine du temps, la methode de l’index de performance et celle 
de la comparaison avec le module peuvent eire ramenees a la forme 
voulue d’une dquation differentielle avec le signal d’erreur comme 
variable. On 6tudie les avantages de ce type de systemes de commande 
adaptative. On donne un exemple et des r£sultats expdrimentaux. 

Zusammenfassung 

Der Aufsatz behandelt ein Verfahren zur Synthese von selbsteinstellen- 
den Regelsystemen, das eine schnelle Selbsteinstellung ermoglicht. Dies 
fiihrt zu einem Modell eines selbsteinstellenden Regelsystems; es 
handelt sich aber nicht um einen Vergleich mit einem Bezugsmodell 
(Modellvergleich). Das Modell dient vielmehr als dominanter Operator 
(der das dynamische Verhalten des Systems beschreibt), daher laBt 
sich das dynamische Verhalten der Regelstrecke vernachlassigen. Es 
wird die Synthese des dominierenden Operators im Zeitbereich einge- 
fiihrt. Bei der Synthese im Zeitbereich lassen sich das Verfahren des 
Giitekriteriums und der Modellvergleich in die gewunschte Form der 
Differentialgleichung fur die Regelabweichung uberfiihren. Die Arbeit 
untersucht die Vorteile dieser Art der selbsteinstellenden Regelung. 
Ein Beispiel und einige Versuchsergebnisse sind angegeben. 


Introduction 

Several different philosophical approaches to defining satis¬ 
factory performance of an adaptive control system have been 
offered; two of the most common are the performance criterion, 
and model comparison. In performance criterion approach some 
generalized performance index, such as the integral of squared 
error, is chosen and is computed continuously. 

This information is used to adjust the proper system para¬ 
meters in the required manner to keep the value of the index at 
a minimum. In model comparison approach a specific model is 


chosen which represents the desired system characteristics. If 
the parameters of the system deviate from those specified by the 
model, they are suitably adjusted. These two general procedures 
are illustrated in Figure 1. 

The principal drawback to the first scheme has been the 
inability to arrive at a performance index which will provide a 
satisfactory system characteristic for all inputs. Another dis¬ 
advantage to the first scheme has been the delay in the per¬ 
formance index control loop. 

The second scheme also, however, presents a fundamental 
obstacle, in that the procedures required to calculate the existing 
system parameters usually involve correlation techniques which 
require integration and averaging of input and output signals 
over periods of time that are long compared with system time 
constants. 

Hence in these situations the process parameters are con¬ 
strained to vary slowly compared with either the system dynamics 
or the input signal, in order that the parameters can be identified 



(a) 



Figure 1. Two classes of adaptive controller: (a) performance index; 
( b) model comparison 
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before they change appreciably. Often this constraint is easily 
met—as in the case of many industrial adaptive control require¬ 
ments but rapid systems are necessary for aircraft and space 
vehicles. 


Dominant Operators 


Fs d t F ° ,+ ^ F °" + - 


where 


+ (-l) k f k F cW - 

dr 



a f 

8s 


( 2 ) 


Ten years ago, Chu 2 proposed synthesis of linear control 
system by predominant roots. In this paper it is assumed that 
all the modes of signal output equation may be neglected except 
die first constant term, due to the pair of predominant roots. 
This approach was used to the synthesis of adaptive system by 
Staffin 9 , who has proposed a system which provides for neglect¬ 
ing process roots. As shown in Figure 2 this is a model system 
but not a model comparison system. A model is used in this 
system as a dominant operator and then dynamics of process 
may be neglected. This approach was used also by Li 4 , Mellen 6 , 
and Schuck 13 . 



Figure 2. Staffin's executive-controlled adaptive system 


iv= 


0F 

de' 


QF 



Se 0 

The general solution of eqn (2) has 2k arbitrary constants, 
which may be found from the boundary points 

(*-i) 

= = e (f 0 ) = a ( 0 k ~" 1} 

(k- i) 

Thus, when eqn (2) is solved the following is obtained: 

In many cases it can be assumed that 

£ 1 = 0 5 6 ' 1 = 0 ,...,s ( 1 fc - 1 >=0 ( 4 ) 

Putting (4) into (3), the following form of eqn (3) is obtained: 



Staffin has proposed j-domain synthesis of a dominant 
operator adaptive system, but this method is useless for rapid 
adaptations because identification of process parameters is 
needed. 

A more useful system for rapid adaptations may be obtained 
when a time-domain synthesis of a dominant operator is used. 
This method has been based on the earlier works of Straszak 10 - 11 . 


e(0 =/* [t, s (t 0 ), s (t 0 ),..., £ (f 0 )] (5) 

Differentiating (5), the following is obtained: 

de(t) r , <*"« 

— o lr> £ (to), £ (1q)> •••> £ (to)] (6) 

Applying the principle of optimality 1 it can be assumed that 


Time-Domain Synthesis of the Dominant Operator 

To keep the value of the performance index at a minimum 
or to obtain the desired system characteristic is the goal of the 
adaptive control system. 

Consider first the minimum performance index system. 
Usually, in control systems the performance index is a function 
of the error signal 14 . Thus, the performance index, K, is 

.£= F[a(t),e(t),..., £ w (f),f]df (1) 

J tz 

Now, it is necessary to find an operator (dynamic system) which 
will minimize the performance index K. 

If it is assumed that a function F is known, k + 2 times the 
differential function of all the arguments, then very well-known 
variational methods may be used to obtain the Euler-Poisson 
equation in the following form. 


f=f 0 =0 (7) 

By putting the problem (7) into form (6), a differential equation 
is obtained: 

de 

£) ( 8 ) 

of dynamic system (operator) which will be minimized by the 
performance index K(e) 

The problem becomes more complicated when the per¬ 
formance index K needs to be minimized, subject to the con¬ 
straints 

(k) 

a , ...,£)=0 

or 

pi (k) 

Fi(t, M,... ,£) = /,- 
J t 2 

but this also may be solved by using Lagrange’s method of 
multipliers. When classical variational methods cannot be used. 
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the methods of Pontriagin, Bellman or Kulikowski 3 may be 
applied. 

When a solution of a variational problem is to be obtained 
a procedure from (3) (8) may be used and then a desired 

differential equation of a dynamic system which will minimize 
the performance index K(e) subject to the constraints is obtained. 

In model comparison approach a specific model is chosen, 
which represents the desired system characteristics. Usually, a 
model has been known in transfer function form, but it is very 
easy to transform a desired system characteristic to the differen¬ 
tial equation form, as (8). 

Thus, from the time-domain synthesis approach, the perform¬ 
ance index approach and model comparison may be trans¬ 
formed into the one form, desired differential equation of error 
signal. 

Now in a general case it can be assumed that the desired 
differential equation in form (8) is given. 

Consider accordingly the adaptive control system with the 
following desired differential equation of error signal. 

d s (w) 

-g=r(e,...,M) (9) 

The form of the physical adaptive control system chosen for 
time-domain synthesis is shown in Figure 3. Although this 



Figure 3. Adaptive control system chosen for time domain synthesis 

form is not essential for the application of the synthesis proce¬ 
dure, the explanation of the operation of the system is greatly 
facilitated, especially for the dominant operator-adaptive con¬ 
trol system. 

Assume that the process may be described by a differential 
equation which is not known, 

dv (m) 

-^=p(y,...,y,u,t) (10) 

but a solution of this equation may be observed. Now we can 
formulate our goal: we want to find a control signal u(t) which 
will give desired system characteristic. 

Consider now the error signal in this system. It may be as¬ 
sumed that the control signal u{t) is not changed for a fixed 
period of time t 0 < t < ^ and observe the error signal as a 
function of time ( Figure 4). On the same figure we may draw the 
solution of the desired differential equation with the same initial 
values in point t 0 . 

Dividing by r time axis, two sequences are obtained: 

K Oo)> £i Oo + 0, 8i (*o + 2 0,.. e t (t t )} 

and 

{s 2 ( t 0 ), e 2 ( t 0 + x), s 2 (to +2 X ),.. e 2 (*i)} 

where 

% is the desired value, and 
e 2 the uncontrollable value. 

Suppose that system has been at t 0 r point. 


At this point, there have been two values e 2 (t 0 + t) and 
H (4 + t). To obtain the desired value only the control value u 
can be changed. 

This value may be changed at random because it is not 
known how much change is necessary. Thus an additional func¬ 
tion is obtained: e Au {t) {Figure 4). 



Figure 4. Errors signals: e ± (t)—desired function; e 2 00 —uncontrollable 
signal; e A u (7 )—error signal response for A ul 


Usually, for most of the processes, e Au (t) is equal to zero at 
the starting point and only this type of process is considered. 

Therefore, to obtain the desired value e 1 (t 0 + t ) the control 
signal u{t) must be changed some time before; for example 
at t 0 . 

Error signal £ (t 0 + r) may be written in the following form. 
e(t 0 + x) = x(t 0 +x)-y(t 0 +x)=x(t 0 )-y(t 0 ) 

+ (S = w) T+ L° h “(r-x)du(x) 

f(o + t 

+ h b (x-x)db(x)±y+R x +R y ... (11) 
J t 0 

where 

h w h b is the response of system to unit step control signal u 
and disturbance signal b, 

A y is the jumping function, and 

R x9 R y the second order small value. 

In eqn (11) there is only one controllable term, as 

h u { t~x) d u (x ) 

J to 

Now, r will be very small, so it may be assumed that 

[*t+T 

ft 6 (T-x)dfc(jc)«0 

J ro 

and 

R X ,Ry* 0 

Jumping function Ay may be removed to the next point. 

Therefore, the necessary increment of control value at t 0 , for 
obtaining the desired error value at t 0 + r, is 11 


A S 1 {t + T)~£ 2 (t + T) 

Au ~ — m — 


( 12 ) 
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This may be rewritten as follows: 

£i(0 d6 t (Q e 2 (t) 
t d t 2 

since 



dt 


where 


s 1 (t)=e 2 (t) 


Au(t) = k 


de, . s d£,.. 

■ 3 r«-ir« 


■] 


(13) 


(14) 


k— 


T 

h(?) 


Now, substituting (9) into (14), gives 


(«) 

Au(t) = k[r(s,...,E,t)-e] (15) 


k is unknown a priori, but assuming this value at random is not 
necessary from the practical point of view. 

_ Usually, it is not very difficult first to estimate maximum and 
minimum values of h u ( r) and then to obtain average value of 
h u ( r) as follows 


h max + fyi( T )min 

n u avr V. -X- 


(16) 


r must be chosen much smaller than time constants of the pro¬ 
cess. The physical realization of this procedure is illustrated in 
Figure 5. 



=(p- i) 
=(/>-!) 


where 

Assume 

then 

For 


de 


a -p)+p 


dt 


dSi \ f d e t 


dt 


t+z. 


LVd^ (1_p) ' 




dsi 
dtj t 

da* 


0 n-p ) 


da j 

V df 7t 

rjttl 


StP=\(p- l)(l-p) 


de\ / de x 


LVd t/ t j 

lpl<2 
|i 5£ ii |<|& i | 


sign (5s 11 = - sign (5a 1 

Now, it can be concluded that this system has self-correcting 
ability; this property was confirmed by experimental study (see 
example). 

If a dynamic characteristic of process is changing too much 
an additional control loop may be used to endeavour to solve the 
following equation 

h(r) . 

~— = k — const (17) 


by adjusting the sampling interval r. Multiplying (17) by 
Au (t-j-r) and x 

Au(t+x)h(T) = k-T-Au(t + r) (18) 

is obtained since 


Figure 5. Dominant operator adaptive control system 


Now, we consider a sensitivity of inaccuracy of gain coeffi¬ 
cient k on the system characteristic. 

Now, let Se = e 1 (/-f-r) — e (? + t) be an adaptive error func¬ 
tion and 


be an inaccuracy of gain coefficient. 
Then 


(t+x)~e(t+x) = 


ds 


= (p — 1) 


|7dA f d s t \ 
L\df/ f \dt J L 




de 


dt , 


dgj 

dT/t+t 


Ss ll = S 1 (t + 2z)-s(l+2t) = (p~l )\(_ 

LV d V t + r 

t+t \dty f l\dt J t \d?y f J 

i+oiteufe) f d£ i\ ii 
* lWr Wrw t+ J_T 


<5£ H =(p-l) 


x{t + x)-x{t)-~-xKAu{t+x)h{x) (19) 

Substituting (19) into (18), gives 

x{t + x)-x{t)-{~jx = k-T-Au{t + x) (20) 

This equation may be solved automatically as shown in Figure 6. 



Figure 6. Dominant operator adaptive control system with additional 
control loop 
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Figure 7. Example of dominant operator adaptive control system 


Example 

As an example an adaptive control system has been studied, 
for which the desired system characteristic e t (f-f-r) has been 
simplified and depends on only one variable e(t). 

Let the performance index K have the form 


K = 


00 

0 


[«(0 


Assume also that there is an additional constraint^, 




Using Euler-Poisson equation and Lagrange’s method of multi¬ 
pliers, gives 

(2) 

— As (t) + s(t) — 0 
Solution of this equation is 

_t_ _ t_ 

Assume that 

s 1 ( c °)=e 1 (T)=0 

where T is the time of transient state. 

Obtain, 

_ 

e i (0 = £ ( t o) e 

and constraint equation gives 

2 Kir 

8l (t+r) = s(t)e~te(0l 2 

Using (12) and (16) we obtain the adaptive control system with 
dominant operators as shown in Figure 7. 

Operation of this system is illustrated in Figures 8 and 9. 
In Figure 8 are shown error signal responses of this system for 
different transfer functions of process: Figure 8 (a), transfer func¬ 
tion of process K 0 /G(s) = K 0 /2s + 1 Figure 8(b) , transfer func¬ 
tion of process K 0 /G(s) = K 0 /6s + 1 Figure 8(c) transfer func¬ 
tion of process K 0 /G(s) — K 0 /s (2 s -+- 1) 

In Figure 9 are shown error signal responses of this system 
for different gain coefficient k and one transfer function of 
process k 0 /G(s) — k 0 /s (2 ^ -f-1) 

Figure 9(a): k x = 3 k 0pt ; Figure 9(b): k x = k opt ; 

Figure (9(c): k x = 1*5 k opt 




Figure 8. Error signal responses for different transfer function of process: 
(a) K 0 /G (s) — KJ2 s + 1; (b) KJG (s) = K 0 /6 5 + 1; 
(c)K 0 IG(s)=K 0 ls(2s ■+• 1) 

Conclusions 

Dominant operator approach to the synthesis of the adap¬ 
tive control system has been presented. Time-domain synthesis 
of a dominant operator has been used. Examples of a dominant 
operator adaptive control system have been given. This tech¬ 
nique supplies a simple method of synthesis for a rapidly 
adapting control system. 
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Figure 9. Error signal response for different gain coefficient and one 
transfer function of process: 

(a) k j = 3 &opt/ 

(6) k x = k opt ; (c) k, = 1-5 /topt 
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DISCUSSION 


A. R. M. Noton, Electrical Engineering Department , University of 

Nottingham, Nottingham, England 

Unless the additional loop of eqn (17) is added, this is surely not an 
adaptive system; there appears to be no adjustment of parameters to 
compensate for either changing process characteristics or changing 
input statistics. 

Furthermore, even if an adaptive loop is included to solve eqn (17) 
(k = h (t)/t) there is no guarantee of stability. Stability cannot be 
guaranteed by using only the information h u (t), i.e. the value of the 
impulse response time r ahead. 


A. Straszak, in reply 

The property of a control system to be adaptive depends on the be¬ 
haviour of the system and not on how many additional loops the 
system has. 

Self-adjusting systems may be adaptive or not, because the adjust¬ 
ment is in a certain sense only the technical realization of some non¬ 
linear equation. I understand by ‘adaptive control system’ a system 
whose behaviour comes close, in an asymptotical way, to the 
optimal behaviour. 

The stability problem for this kind of system must be defined in a 
different way than for conventional control systems, because with 
these systems we are interested in its working performance in a finite 
interval of time, which indeed is usually short. 


M. Hamza, E.T.H. Zurich, Switzerland 

I read the paper of Dr. Straszak with great interest. In the paper two 
systems are considered, that of Figure 5 and of Figure 6. To obtain 
Figure 5 many approximations were made. I would like Dr. Straszak 
to comment on the performance of this system as compared with the 
types of adaptive control systems referred to at the beginning of his 
paper. Further, I feel that the system of Figure 6 will have a long system 
response time or its performance will not be better than that of the 
methods referred to in the literature (model and performance index). 

A. Straszak, in reply 

In my paper two different adaptive control systems are presented, but 
only the first of them (Figure d) is a very fast adaptive control system. 

o obtain fast adaptation, it is necessary to use the dominant operator 
approach or a similar technique, because in most approaches we lose 

f ° r measuring the index Performance or for comparison 
with the linear model (usually transfer function). 

Published work shows that the additional adaptive loop must be 
about 1,000 times slower than the main control loop in conventional 
adaptive control systems. 

A. M. Hopkin, University of California, Berkeley 4, California, U.S.A. 

Not everyone agrees as to the precise meaning of the term ‘adaptive’. 
Professor Lotfi Zadeh has defined an adaptive control system as one 
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designed by a person with an adaptive viewpoint. By the Zadeh defini¬ 
tion the system under consideration is undoubtedly adaptive. 

However, a person accustomed to non-linear controllers sees here 
a fixed, non-linear control system with sample-and-hold, using feed¬ 
back to minimize the overall system sensitivity to variations of process 
dynamics. Originally, linear feedback was introduced for precisely the 
same purpose. The test of any controller is its degree of success in 
maintaining the desired output in the face of process variation and 
external disturbances. 

The author defines the desired output in terms of an error criterion 
with possible constraints, independent of the process dynamics. This 
implies that the ideal response should be independent of variations of 
process dynamics also. A measure of the success of the design is the 
closeness of matching of the actual and ideal responses under various 
conditions of process variation. Figure 8 of the paper shows consider¬ 
able variation of system response with variation of process dynamics. 
No information is given to permit comparison either with the ideal 
performance pattern or with behaviour of other controllers that might 
have been used. 

(1) Does the author have comparison data to show that under 
similar environment this system response is much better than a simpler 
system, or that this system behaves almost as well as a more elaborate 
system? 


It has been my experience that non-linear control systems have 
different forms of response for different initial states. In some cases 
the response will be quite satisfactory for one set of initial error states, 
but not satisfactory for another set. 

(2) How does the example system behave for various initial states, 
for example various initial error amplitudes ? 

A. Straszak, in reply 

It is well known that there exist almost a hundred different definitions 
of the term ‘adaptive’; therefore I will not attempt to discuss this 
problem again. 

In the very simple example which I presented for illustration only, 
the maximum difference between the ideal control system and the 
dominant operator adaptive control system was less than 5 per cent 
for different controlled plants. 

The dominant operator approach to the synthesis of adaptive 
systems is based on the state-space technique, therefore this approach 
is good for all initial states, provided the dimension of the control 
vector is equal to the dimension of the state vector. This does not 
follow from the paper as I presented it to this Congress, but from the 
extension which I have mentioned at this meeting. When we use so- 
called pseudo-states, then the problem concerning the initial state has 
to be studied very carefully. 
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Summary 

The stability and response characteristics of extrema seeking adaptive 
systems, in which sinusoidal perturbations of the adaptive parameter 
(parameters) provide the derivative action necessary to determine 
direction and magnitude of the criterion surface gradient at the 
operating point, are analysed. It is assumed that system variations 
with time occur slowly as compared with the adaptive loop response 
time, and that the measured criterion output may be characterized 
by a general function of the adaptive parameters (usually non-linear) 
followed by a linear transfer function representing measurement delay. 
The resulting adaptive control loop is analysed, using modified band¬ 
pass to low pass transformation techniques, yielding an equivalent 
low pass loop in which no perturbation signals appear. This loop is 
generally non-linear but standard techniques, such as the describing 
function, may be applied to predict the critical loop gain, the frequency 
and amplitude of the resulting oscillations, and the required equal¬ 
ization. 

Several examples are presented which have been simulated on a 
Pace analogue computer. Predicted and actual results generally com¬ 
pare within 10 per cent, which is within the usual range of describing 
function accuracy. 

Sommaire 

La stabilite et la reponse de tels systemes sont analysees. Le systeme 
utilise des perturbations sinusoidales du ou des parametres adaptatifs, 
fournissant faction differentielle necessaire pour determiner la direction 
et f amplitude du gradient de la fonction-critere. On admet que les 
variations du systeme sont lentes par rapport au temps de reponse 
de la boucle adaptative^ de commande et que la fonction-critere me- 
suree k la sortie, peut etre representee par une fonction generate des 
parametres adapta ifs (generalement non-lineaire), suivie d’une fonc¬ 
tion de transfert lkteaire, due au delai de mesure. La boucle adaptafive 
de commande d’un tel systeme est analysee au moyen d’une theorie 
modiftee de la transformation passe-bande/passe-bas. 

Cette analyse conduit a une boucle passe-bas dans laquelle le 
signal de perturbation n’apparait pas. 

Cette boucle est en general non-lineaire mais des techniques 
normal;sees telles que la fonction descriptive peuvent etre utilisees 
afin de predire le gain critique, la frequence et 1’amplitude des oscil¬ 
lations resultantes ainsi que la compensation necessaire. 

Plusieurs examples, ayant ete simules sur une machine analogique 
PACE, sont pr6sentes et leurs resultats compares avec ceux obtenus 
par la theorie precitee. L’erreur reste dans les 10%, domaine normal 
de precision d’une fonction descriptive. 

Zusammenfassung 

Die Arbeit untersucht die Stability und die Ubergangsfunktion von 
Sy&emen, die selbsttatig nach einem Extremwert suchen. Die sinus- 
fornuge Stoning der selbsteinstellenden GroBen ruft eine differentielle 
Wirkung hervor, die notwendig ist, urn Richtung und GroBe des 
Gradienten der (Kriteriums-)Kennflache im Arbeitspunkt festzulegen 
Dabei wird einmal angenommen, daB sich das System, verglichen mit 
der Ubergangszeit des selbsteinstellenden Kreises, nur langsam 
andert, und zum andem, daB die gemessene AusgangsgroBe durch 


eine (meistens nichtlineare) allgemeine Funktion der selbsteinstellen¬ 
den GroBen und einer nachfolgenden linearen Ubertragungsfunktion, 
welche der gemessenen Verzogerung entspricht, darstellbar ist. Der 
sich ergebende selbsteinstellende Ruckfiihrzweig wird untersucht; 
durch die Abwandlung eines Bandpasses in einen TiefpaB ergibt sich 
ein aquivalenter TiefpaBzweig, in dem keine Storsignale auftreten. 
Dieser Zweig ist im allgemeinen nichtlinear, jedoch lassen sich die 
bekannten Methoden, wie z. B. die Beschreibungsfunktion, zur Vor- 
herbestimmung der kritischen Kreisverstarkung, der Frequenz sowie 
der GroBe der auftretenden Schwingungen und der erforderlichen 
Kompensation anwenden. 

Der Aufsatz enthalt einige auf einem PACE-Analogrechner nach- 
gebildete Beispiele. Die Ergebnisse der analytischen und maschinellen 
Berechnung liegen im allgemeinen innerhalb der Genauigkeit von 
10%, was dem Genauigkeitsbereich der Beschreibungsfunktion ent¬ 
spricht. 


Introduction 

Although adaptive systems have received much attention during 
the past decade, analysis of their stability and response charac¬ 
teristics has often been ignored. There are, however, several 
specific stability analyses available. Morosanov 1 uses the 
describing function to analyse relay operated peak sensing 
adaptive controllers; Margolis and Leondes 2 consider the 
stability of a class of model reference adaptive systems; Schiewe 3 
investigates the stability of a three-dimensional adaptive system, 
assuming the three adaptive loops to be independent and Gibson 
and Meditch 4 discuss the stability of signal synthesis predictive 
controllers. The stability and response characteristics of sinus¬ 
oidal perturbation adaptive systems (hereafter abbreviated 
SPAS) have been investigated by the author 5 * 14 > 15 , and the 
analytical part of that work is presented in this paper. 

The study of SPAS is perhaps as old as adaptive control 
itself. Draper and Li 6 , in a pioneer work, consider an application 
in which spark timing and fuel mixture were controlled to maxi¬ 
mize manifold vacuum, a readily measured even function index of 
performance (IP). The general concept of an even function IP 1G 
is used here to denote any general performance criterion, such 
as R.M.S. error, which exhibits an extreme value for a specific 
parameter combination. McGrath and Rideout 7 discuss a 
system in which two parameters are simultaneously adapted to 
minimize mean square error. McGrath, Rajaraman and Rideout 8 
discuss modifications of this approach, using a model to prevent 
performance degradation, or hunting loss 9 , due to parameter 
perturbations. These references provide an excellent background 
for the subject presented in this paper. 

Since all analytical studies depend upon a mathematical 
system description, it is the purpose of this paper to develop a 
mathematical model of the general adaptive system and proceed 
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to simplify that model until an equivalent block diagram is 
obtained. The block diagram is then used to determine adaptive 
loop stability and transient response characteristics. 


The Sinusoidal Perturbation Adaptive System (SPAS) 

In the interest of brevity only minimum seeking systems are 
considered here, although the SPAS approach applies equally 
well to maximum or minimum seeking systems. Assume a 
one-dimensional SPAS as shown in Figure 1. The first step in the 
analysis of this system is to establish a circuit model for the 
transfer between parameter input and measured IP output, 
denoted by H ( s , x, t ). In general, steady-state IP variations for 
parameter settings near the optimum value are characterized by 
a smooth curve, which may often be assumed parabolic 1 * 5 » 9-13 . 
Variations in measured IP value are not immediately apparent 
when a parameter adjustment is made, but approach the new 
value with delay depending upon the measurement process. 


CONTROL 

LOOP 



Figure 1. The relationship between the parameter adaptive loop and the 
standard controller with closed loop transfer (s) 

Assume that the IP measurement process may be represented 
by a general function of a parameter x, /'(x), in series with a 
linear filter representing measurement lag 5 * 7 . /(x) is assumed 
to possess all derivatives and must provide a well-defined 
minimum point if the system is to operate properly. Assume that 
/ (x) is monotonically increasing as x proceeds from the optimum 
value in either direction. Thus, if x 0 is the optimum parameter 
value (yields minimum IP), 

/'(*)> 0, X>X 0 m 

f'(x)< 0, x<x 0 

where/'(x) denotes the first derivative of / (x) with respect to x. 
An / (x) curve meeting these requirements is shown in Figure 2. 
In practice, of course, the minimum IP and optimum parameter 
values are not fixed, varying with environment, but this does 


CONTROL SYSTEM IP 
MEASUREMENT CHARACTERISTICS 



Figure 3. Block diagram representation of a single-dimensional adaptive 

controller 


not appreciably modify the following analysis as long as their 
time rates of change are small compared to the adaptive response 
rate, which must be true if the adaptive loop is adequate to 
compensate for environmental changes. 

Figure 5 is a block diagram of the adaptive control loop. 
Similar adaptive controllers are discussed in detail in the 
literature 5-7 ’ 14 , so only a brief steady-state analysis of its 
operation is presented, p (t), as illustrated in Figure 2, produces 
y x (t) with an co x component either in phase with, or 180° out 
of phase with, p (t ), depending upon the sign of the parameter 
error. y 3 (t) is an attenuated and phase shifted version of y x if). 
G 3 (s) is chosen so as to give the same steady state-phase shift 
at co x as the combination of G m and G x . Multiplier M x is thus 
a correlation detector, and its average output is either + or — 
depending upon whether y 3 (t) and y 4 (t) are in phase or 180° 
out of phase. This, in turn, depends upon whether the offset 
in x from x 0 is + or —, respectively. G 2 (s) and the integrator 
smooth y 5 (t) and develop a correction signal to reduce the error 
in x. G x (s) and G % (s) are chosen to make the loop best con¬ 
form to design requirements 5 . 

Consider an open-loop analysis assuming the loop is broken 
at the integrator input. Let 

x = x x -\-C x $inco x t (2) 


where x x is an arbitrary initial parameter value and C x sin a) x t 
is the perturbation signal necessary to make the system operate 
using an even IP function. Using a Taylor Series approximation 
for the /(x) curve about x = x x , y x ( t ) becomes 




'(^i) C t sir 


+’ 


/" Oi) 


2 ! 


C x sin co x t 


+ . 


If the /(x) curve is ‘smooth’ and C x is small, 


(3) 


IP 

VALUE 



Figure 2. A representative steady state IP (performance criterion) 
variation about the desired operating point , x = x 0 , illustrating the 
effect of the perturbation signal for + and — parameter errors 


/”(* i) 

n\ 


[C x sin cojfl" 


will be negligible for n> 3. Assume that this is the case. Then 
only the first three terms of (3) need be considered. The third 
term in (3) may be reduced to a d.c. and a 2 co x component. 
The loop can only pass signals at or near o) l9 due to the charac¬ 
teristics of G x and M x , so it is only necessary to work with the 
second (o^) term of y x (t) 5 > 1 > 14 . Assuming that Gm and G x are 

y 3f (t)=C 1 f(x 1 )Asm((o 1 t+9 1 ) (4) 

where A and 6 X are the steady-state attenuation and phase shift, 
respectively, at co x due to G m and G x . The subscript / denotes that 
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only the fundamental term has been considered. G z is chosen 
such that 

y 4 (t) = Bsm(co 1 t + Q 1 ) (5) 

so 

ysf (0=[i ~ cos2(Q) 1 f+ ej] (6) 

The second term in brackets in (6) will be highly attenuated by G 2 
and the integration which follows, or 

ysS - = ABC^ (?) 

where the bar denotes time average. ABC x is positive, so y 6 will 
tend to reduce parameter offset when the loop is closed, due to 
the limits imposed by (1) on f (x). If / (x) is parabolic near the 
optimum, it is easily shown 5 ’ 7 > 14 that (7) reduces to 


y 6 ^y 5 = a(x 1 -x 0 ) (8) 

where a is the gain, indicating that loop operation is quasi-linear 
for a parabolic IP function. In practice, it should always be 
possible to approximate the IP variation as accurately as desired 
in some s neighbourhood of the optimum by a parabolic curve 
(surface in the ra-dimensional case). Thus, if one is interested only 
in defining the small scale asymptotic stability and response 
conditions of the loop, the analysis is considerably simplified. 

The previous analysis was carried out assuming the loop to 
be open so that steady-state sinusoidal techniques could be 
applied. When the loop is closed this analysis is no longer 
strictly valid. If, however, loop response is slow, relative to the 
time required to establish steady-state conditions, as it normally 
is in practice 5 ’ 7 , this analysis closely approximates the actual 
performance. In any event, it provides a heuristic argument to 
explain why the loop operates satisfactorily. 

Derivation of an Equivalent Low Pass Adaptive Loop 

Modified describing function and band-pass to low-pass 
transformation techniques may be applied to the adaptive loop 
of Figure 3 to derive an equivalent low pass network. This is the 
key step in stability study of SPAS. The perturbation signal is 
treated as a carrier, and equivalent low pass networks are 
derived for each of the linear and non-linear loop elements 
between perturbation input (modulation) and correlation 
detector output (demodulation). 

Let 

* (0=* 0 + C t sin co x t + C m sin ( co m t + 9 m ) (9) 

where C m , co m and 0 m are the amplitude, frequency and phase, 
respectively, of a circulating natural frequency component. 
The co m term is the sinusoidal excitation term normally assumed 
in describing function analysis. Assume that co m <4 co x , which 
is true if adequate smoothing (G 1 and G 2 ) is used to assure 
reasonable loop transient response 5 ’ 14 . (The analysis, as 
illustrated in Tables 1 and 2, yields accurate results even as 
co m co x , which represents a severe test of the system’s equivalent 
network derived herein.) The co x component of y x (t) is obtained 
from (2) and (3), assuming that C m sin (co m t -f 6 m ) changes but 
little during one cycle of sin co x 1, as 

y lf (t)=C 1 f' [x 0 + C m sin (.coJ+6 J] sin coj (10) 


C 1 f [x 0 + C m sin (co m t + 0 W )] product modulates the sin co x t 
(carrier) term. The modulation envelope has a frequency com¬ 
ponent at a> m , and its amplitude may be determined by finding 
the Fourier coefficient of the output co m term which results if 
x 0 + C m sin (co m t + 6 m ) is applied to /' (x). Evaluation of 
A (CJ, the describing function for /' (x), is thus the first step 
in development of the equivalent low pass loop. Then 


y if (t) = A (C m ) sin co i f sin (co m i+0 m ) (11) 

or 

y lf (t) = A( - C ^ Cl {cos [K CO m ) t 0J 

—cos [(m x + co m ) t + 9 m ]} (12) 


Bandpass to low pass transformation techniques may now 
be applied to find equivalent low pass filters for G x and G m . 
As an example of this process, assume that 


Gm( s ) = 


K m 


and 




cols 




(13) 

(14) 


These are reasonable choices in the light of experimental 
evidence 5 ’ 14 , but the approach is by no means limited to these 
filters. G 3 may now be chosen as 


G 3 (s) = 


V + l 


(15) 


since G 1 gives no phase shift at co v Any G z giving the same phase 
shift could be used. 

Let the phase shift of G m (j co x ) be 0 l9 and assume that the 
phase characteristic near co x is sufficiently linear that the phase 
shift at s = j (ft) x — co m ) may be represented by d x + A 0 and 
that at j = j (coj + co m ) by 6 1 — A 0. Let the gain of G m (j <u x ) 
= a v Thus 


a i = 


K n 


[1 +(t»o i) 2 ]* 


(16) 


Assume further that the attenuation at s = j (co x — co m ) and 
s — j (co x + co m ) is approximately a x . This is reasonable for 
co m < co x . The steady-state output of G m (s) thus becomes 


y 2 ( t ) = -A ^ Vf ) - {COS l(a> 1 -CO m )t-d m + e i + Ad'] 

- cos [(cox + co m ) t + d m + d x — A(f\} (17) 

which may be rewritten as 


y 2 (0 = a x A (CJ C x [sin (<x> x t + 9 X ) sin (co m t + 9 m -A0)] (18) 


G m thus shifts the carrier 0 X rad and the intelligence signal 
— Ad rad. Only the intelligence signal shift is significant in the 
equivalent low pass loop. 

An equivalent transfer function may now be defined such as 
to approximate the phase characteristics of the original network, 
or 


G m 


as-hi 


(19) 
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oc is chosen such that 

d Qe 
d co 

Thus the phase slope of the low pass equivalent network about 
co = 0 must approximate that of the carrier network about 
co = co v It should be noted that the approach used here is 
somewhat more general than the usual bandpass to low pass 
transformation, applying to all networks whether band centred 
at co x or not. Of course the accuracy is best when dealing with 
high Q bandpass networks centred at co v 

Derivation of G le (,s ) is carried out using similar techniques, 
so need not be discussed in detail. The approach applies even 
if G x (s) is not band centred, as illustrated later with an example. 
The equivalent loop may be revised, if desired, to obtain greater 
accuracy by using a value of A co close to the natural frequency 
of the system to determine the value of oc in the equivalent low 
pass network, i.e., instead of evaluating oc from (20), use 

_A0_ 
dco #B(#1 

where A co is near the natural frequency of the loop. This has 
been found to yield a significant increase in accuracy. 

The phase reference input to M 1 is given by 

y 4 (t) = B sin (co x t + 6 x ) (21) 

and the signal input is 

y 3 / (0 ■= A* sin (co 1 1+ 0 t ) sin (co m t+d m -y) (22) 

where /u is the amplitude, 6 1 and y the phase lags, which appear 
after passing through G m and G v Multiplying (21) by (22) gives 
the M x output as 

y S / (0=juB sin 2 (ciV + sin (coj + 9 m -y) (23) 

which may be rewritten as 

y 5 / (0=^r sin - y) [1 - cos 2 (ft)! t + 0j)] (24) 

The first term obtained by expanding (24) is the desired signal 
and the second is a relatively insignificant high frequency 
component. The describing function for M x is the ratio of signal 
out to envelope amplitude in, or 

Gmi-y (25) 

The remaining elements in the loop act upon the signal terms 
directly, so no further transformations are required. 

Example 1 
Let 

C 1 = oo 1 = 10, T m = 0*1, £ = 0-707 and K m — 1 
Assume that 

f(x')=IP = k 1 (x-x 0 ) 2 + k 2 (26) 

adequately approximates the steady-state IP curve near the 
optimum. Then 

f'(x)=2k 1 (x-x 0 ) (27) 

and (10) becomes 

y lf ( t ) =2k 1 C m C 1 sinco 1 t sin (co m t+6 m ) (28) 




co = 0 


d0 

dco 


( 20 ) 


A (C m ) is thus 2 k x C m and the ‘gain’ associated with /(x) is 
2 C x k x ~ 20 k v Letting k x = 0*01, 20 k x — 0*2. oc is found 
from (20) to be 0-05 and a x = 0*707, so 


G me ( s ) = 

Similar calculations show that 
G ie (s)= 


Combining these equivalent low pass networks yields a 
loop transfer 

0-5 KG 2 ( 5 ) 


0-707 

(29) 

0-05 s + 1 

10 

(30) 

0-2s + l 


G Le (s) = 


s (0-05 5 + 1) (0-2 s+1) 


(31) 


This Gh e (s) has been used to predict critical gain levels K u and 
oscillation frequencies co 0 for various choices of G 2 (s). The 
results are compared, in Table i, to simulation results obtained 


Table 1. Predicted versus actual instability conditions . K u is the critical 
value of the K defined in Figure 3. G\(s) is as defined by eqn (14) with 
co 1 — 10 in each case 


G 2 (s) 

COi 

Predicted 

Actual 

co 0 

K u 

(Oq 

K u 

1 

(0-2 s + l) 2 

10 

2*8 

8*9 

3 

8*5 

1 

(0-5 j + l) 2 

10 

1*45 

4*6 

1*47 

4*8 

1 

(0-2 5 + l) 2 

20 

3*5 

22 

3*92 

25 

1 

(0-5 s + l ) 2 

20 

1*7 

12 

1*97 

14 


using an analogue computer. Results presented for co x = 20- 
were obtained by deriving new equivalent transfers for G m (s) 
and G x (s), but G x (s) was left centred at co x = 10. This represents 
a severe test of the general approach. 

Example 2 

Consider an IP variation defined as follows: 

f(x)=JgX 2 —25<x<25 

f( x ) = |jc| —12-5 |x| >25 

For this IP function the carrier signal amplitude increases 
linearly with parameter error until the error reaches ± 25 and 
remains constant as error increases beyond that point. The 
equivalent loop for C x — co x = 10 and r m = 0*1 is shown in 
Figure 4. f' (x) is the standard saturation function, as readily 
derived from (32). 

Predicted and simulated values of critical gain and natural 
frequency for this system are compared in Table 2. Figure 5 shows 
curves comparing predicted and actual limit cycle amplitudes 
versus K for three choices of G 2 (s). 
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x 


Figure 4. The equivalent low pass loop for Example 2 


Table 2 . Predicted versus actual instability conditions for the network 
shown in Figure 4 



Gain = K 


Figure 5. Comparison of predicted and actual limit cycle amplitudes 
for Example 2 

Conclusions 

A general method for determining the stability and response 
characteristics of single-dimensional SPAS has been developed. 
This method depends only upon the availability of a mathemat¬ 
ical or graphical approximation of the IP function used to 
develop the adaptive error signals. A method for deriving an 
equivalent low pass control loop has been outlined and examples 
presented to illustrate its use. If the IP is parabolic in the 
equivalent loop is linear, but a series non-linear element is 
required to represent more complex IP functions. The method 
has been tested using analogue computer simulations which 
show predicted and simulated results to agree within 10 per cent 
in most cases. The method thereby yields results within the 
accuracy normally anticipated from non-linear analysis tech¬ 
niques such as the describing function. 


Derivation of the equivalent low pass has been emphasized 
in this paper. Additional analyses use standard techniques, so 
have not been discussed in detail. The method is clear. Determine 
or approximate /' (x) in the region of interest. (A straight line 
approximation should usually be adequate.) Find the describing 
function for this f'(x). Determine the equivalent low pass 
networks for G m and G v Determine the detector gain. Analyse 
the resulting control loop. 

In those multi-dimensional cases where there is little inter¬ 
coupling between the adaptive loops, this approach may be 
applied to each of the loops in turn to predict their response 
characteristics and instability conditions. Allowance of some¬ 
what more than the usual gain and phase margins in the individ¬ 
ual loops generally yields satisfactory overall multi-dimensional 
adaptive response. Loop intercoupling can be minimized by 
using narrow band filters for the n G 1 (s) units required. Narrow 
band filters give more phase lag and thus yield slower response, 
so a compromise is required. Intercoupling may also be reduced 
by careful choice of the adaptive parameters. No general method 
has yet been developed for the consideration of ^-dimensional 
systems with significant intercoupling between the adaptive 
loops. This problem deserves further attention. 

Although no justification is attempted, the equivalent low 
pass loop derived herein may be used as a guide for design 
equalization of SPAS loops to give a desired damping factor, 
peak overshoot, bandwidth, etc. Further work is being done in 
these areas and many interesting problems have yet to be 
considered. 
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DISCUSSION 


O. L. R. Jacobs, Kings Buildings , Mayfield Road , Edinburgh 9, Scotland V. W. Eveleigh, in reply 


The author has presented an interesting approach to the analysis of 
sinusoidal perturbation extremum control systems. 

In recent years a variety of extremum control systems have been 
described 1 , and one outstanding problem is to compare their perform¬ 
ances. For this purpose the general formulation of an extremum 
control system shown in Figure A might be adopted. 


Adjusted 

parameter 


Disturbance 



Controller 


Figure A. Extremum control system (q, q', z may be vector quantities) 


In order to avoid triviality in certain examples it is necessary for 
the general formulation to include three factors: random disturbances 
z which make continuously acting extremum control necessary; and 
random measurement noise £ and dynamic lags which are the two 
factors limiting performance. 

Controllers that have been described in the literature use extremum 
control strategies that fall into two categories; those using perturba¬ 
tions such as described here, and those using finite steps 2 such as 
described in the paper by J. L. Douce. Intuitive reasoning suggests 
that a perturbation strategy should be used where noise is the dominant 
factor limiting performance, and that a stepping strategy should be 
used where dynamic lags dominate performance. Unfortunately it 
has not yet been possible to make any comparison for the two types 
of strategy controlling the same system. 

Published analyses of extremum control systems tend to neglect at 
least one, and often two, of the important factors (disturbances, noise, 
lags). Analysis of perturbation systems often neglects noise, analysis 
of stepping systems often neglects dynamics, analysis of both types 
often neglects disturbances. 

In the present paper both noise and disturbances are neglected, and 
one output lag is considered. I would like to ask the author whether he 
thinks it possible to extend his approach to analyse the performance 
of a perturbation system in which noise and disturbances are included, 
so that it may be possible to make comparisons with stepping systems. 

References 
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I thank Dr. Jacobs for his interesting comments and his question 
regarding the effect of noise upon loop performance. Noise effects 
have, in a limited sense, been considered in the analysis, since they 
determine the smoothing time constants required in the IP measure¬ 
ment circuit. This is not the sense referred to by Dr. Jacobs, however. 

I feel that the analysis can be extended to include noise and transient 
disturbances, thereby allowing a fuller comparison with stepping 
systems. In fact, a comparison between the transient response charac¬ 
teristics of SPAS and discrete systems has been carried out (see Refer¬ 
ence 5 of the paper) using a model IP surface, measurement lag, and 
disturbances. Only a limited noise analysis was carried out, however. 
Our current plans are for further study of this problem, including 
recent developments in the technology, and I hope to report the 
results in detail at a later conference. 

J. D. Roberts, 77, High Street , Cherry Hinton, Cambridge, England 

The stability of an SPAS is very important and it is significant that 
such realistic results have been obtained with this theory. It must be 
mentioned, however, that stability investigations need to be different 
in linear and non-linear systems. A linear system, if it is asymptotically 
stable in one state, will converge to that state from any state. A non¬ 
linear system may, however, have bounded regions of stability in its 
phase space. If an SPAS is to respond to Gaussian random disturb¬ 
ances, it must be stable for all values of its phase coordinates; but no 
value of K in a simple SPAS system of the type described makes the 
system stable everywhere; as shown in Figure A. 



The system has two phase coordinates 6, x which obey the following 
equations: 


= 0Qi 


d9 
d t 

-jKa sin0(a sin0 + x) 2 
at 


Omitting the most tedious part of the mathematics, we note merely 
that a differential equation of the form 

^=/0)x 2 where f(t)>f 0 >0 


All 










V. W. EVELEIGH 


is Laplace unstable, i.e. it goes to oo in a finite time. Moreover, this 
finite time can be made arbitrarily small by making the initial value 
of x sufficiently large. Since sin 6 is positive for a finite time, the system 
is unstable for some phase coordinates for any value of K. 

It would be ideal to have an SPAS which was stable everywhere. 

An SPAS of the type shown in Figure B will converge to an unknown 
hill of known unit curvature from any point of the phase coordinates. 


Unknown hill 
-V 2 u 2 +Xu +Y 



Figure B 


We have the equations: 


computer hard into saturation, which is similar to the phenomenon 
suggested by Dr. Roberts. In all cases considered, however, it was 
found that a reasonably large region in the parameter space was 
unconditionally stable for properly chosen gain levels. 

J. L. Douce, Queen's University , Belfast , N. Ireland 

I disagree strongly with the analysis presented in this paper, applied to 
practical optimizing control systems. The author assumes that per¬ 
formance index, such as R.M.S. error, can be related to parameter 
value by a non-linear characteristic followed by a linear filter. This 
is not so, except in the case of theoretical interest only, when the error 
is a constant with no variation with time whatsoever. In any control 
system the error fluctuates, and to obtain any measure of the change 
in performance index due to a parameter perturbation we must, 
from statistical limitations, measure R.M.S. error over a time period 
very much longer than the response time of the system. 

The author’s values used in the examples are quite unrealistic. 
The theory applies more to the analysis of particular non-linear 
feedback systems than to self-optimizing control systems. 

I agree with the author that the response time of this system is 
slow. In practice, however, increasing the gain produces oscillations 
at perturbation frequency, not at some unrelated lower frequency. 
If a clamp or sample and hold is inserted in the optimizing loop, the 
gain can be increased by a factor of about 30, then the speed of response 
is correspondingly increased, and parameter fluctuations eliminated. 
Stability analysis of this new system is relatively simple. 

As a minor point, the author suggests that the phase lag of the 
control system is cancelled by a network G z (jco). With high perturba¬ 
tion frequencies this is just not possible, since the phase shift in the 
control system varies with parameter value. Incorrect phasing causes 
the system to settle to a false minimum, and can give instability in 
extreme cases. 


u = x + asin co x t 


V. W. Eveleigh, in reply 


& == (X —x) u + Y — y 
dx 

-j— =Keu 
at 

dy 
d t 

Now consider the Liapunov function 

L = (X-x) 2 + (Y-y) 2 


which satisfies: 


= -2Ke\_(X-x)u+Y-y~\ 
= — 2Ke 2 <0 


V. W. Eveleigh, in reply 

I thank Dr. Roberts for his interesting example illustrating potential 
instability regions for adaptive systems of the type discussed in this 
paper. I agree that such systems, depending upon the form of the IP 
surface, are often unstable for sufficiently large initial conditions. 
This defines a region in the phase space which, it is interesting to note, 
because of the sinusoidal perturbation signal, has time-varying 
boundaries outside of which the system is unstable. It was found in 
some of the experimental work upon which the paper is based that 
attempts to initiate action from large values of x often drove the 


I thank Dr. Douce for his list of comments which will help me to answer 
questions that may have arisen in other people’s minds as well. First 
of all, regarding the representation of IP measurement process as a 
general non-linear function in series with a linear filter, this approxi¬ 
mation is consistent with the usual methods used in the literature. 
It is assumed throughout the development that the IP function is a 
random variable with expected value in steady state as described by 
the non-linear curve. The filter is included to represent the smoothing 
required to yield a reasonably steady output approximating the mean 
value of this random variable, which may be R.M.S. error, manifold 
vacuum, or some similar signal. It is true that, for the usual IP's and 
IP measurement techniques used, the measurement time constant 
(or time constants) is significantly longer than the time constants of the 
control loop itself. Thus adaptive loop response is generally slow 
relative to that of the control loop. I feel that this representation of a 
practical IP measuring device acting upon manifold vacuum, for 
example, is quite reasonable. Thus the practical significance of the 
work would not seem to be appreciably degraded by this assumption. 

The effect of noise at the measured IP output upon system perform¬ 
ance may readily be determined, either analytically or experimentally. 
If the noise is uncorrelated with the perturbation signal, it has no 
average effect upon loop response. The parameter value will be dis¬ 
turbed in an R.M.S. sense from its optimum setting by an amount 
proportional to the noise level. However, this does not alter the general 
stability characteristics of the loop in most cases of interest and par¬ 
ticularly not in either example presented in the paper. 

The values used in the examples were chosen primarily for con¬ 
venience in the computer simulation of the system, and not with a 
specific application in mind. 

It is true that in many cases increasing adaptive loop gain suffi¬ 
ciently will produce oscillations at the perturbation frequency, but, 
as illustrated by the data from Examples 1 and 2 in the paper, 
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oscillations at lower frequencies often occur for much lower loop gain 
values. The analysis predicts accurately the frequency of this oscilla¬ 
tion and the gain at which it begins. As pointed out by Dr. Douce, 
the use of a sample and hold filter offers interesting possibilities for 
reducing response time and improving loop stability. 

I see no reason why the response lag of the IP measurement pro¬ 
cess should consistently vary with parameter value, it generally depend¬ 
ing primarily upon the time constants chosen for the smoothing 
filters used to extract an approximation to the mean value of the IP 
function. If it does change, however, due to some other phenomenon, 
I can only suggest that G z ( jco ) should be chosen to give the best 
compromise phase shift. 

Finally, the limit cycle exhibited in Example 2 is a direct result of 
the saturation non-linearity and frequency characteristics which appear 
in the low pass equivalent loop for that example, shown in Figure 4 
of the paper. Standard analysis of that loop using describing function 
techniques yields limit cycle amplitude as a function of loop gain, as 
shown in Figure 5. 


R. K. Smyth, 1960 Domingo Road , Fullerton , California , U.S.A. 

Dr. Eveleigh presents an interesting method for deriving the equivalent 
low pass transfer function for the bandpass filter contained in the 
adaptive adjustment loop. 

The discusser has considered a similar problem and has derived a 
somewhat different approximation which also gives good experimental 
correlation with the predicted stability. The bandpass filter considered 
was: 

2 Cay 

s 2 +2 £ co n s + col 

which is similar to the bandpass filter in the paper’s eqn (14). 

For this transfer function it was shown that a good approximation 
for the low pass characteristics of the bandpass filter is given by: 


G le 0) = 


!>„ a.£co„ 
s + Cco„ s+a£co n 


where oc = 4 for low values of the damping ratio £. Note that the first 
term is the one obtained with conventional a. c. servo theory. 

Would the author like to comment on this problem? 


Reference 

Smyth, R. K. and Nahi, N. E. Phase and amplitude sinusoidal dither 
adaptive control system. Proc. Jacc. (June 1963) Minneapolis, Min. 
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V. W. Eveleigh, in reply 

I thank Dr. Smyth for his suggestion of an alternate representation of 
the bandpass filter to more accurately account for the lack of arith¬ 
metic symmetry about the centre frequency. 

Dr. Smyth’s suggestion may provide an efficient way to improve 
accuracy while bypassing the iteration required in the incremental 
improvement technique presented in the paper. I plan to compare the 
resulting accuracies of these approaches. I would like to mention that 
the low pass filter G m ($) is likewise asymmetrical about the perturba¬ 
tion (carrier) frequency and gives rise to similar difficulties. 


P. Eykhoff, Technological University, Electrical Engineering Depart¬ 
ment, Delft, Netherlands 

Dr. Eveleigh discusses the case of an optimum searching system based 
on the relation 

d{IP) Jd(IP) _ d(IP) dx 
dt dt dx di 


where, in his notation, IP is the measured index of performance and x 
is the parameter (set point) to be adjusted. 

If dxjdt is a known test signal and if d (IP)jdt can be measured 
then information on the ‘gradient’ 5 (IP) fix can be derived. 

The study of these types of systems is a timely subject. This is 
indicated by the fact that at this conference, besides the paper at hand, 
there are also two other presentations, by Alimov, and Isobe and 
Totani, that discuss such topics. 

In this discussion I would like to indicate briefly another approach 
to the analysis problem 1 , 2 . This approach has the advantage that 
it does not require the assumption made in connection with eqn (9), 
which amounts to a rather slow convergence of the adjusting scheme. 


Disturbance 

gu(0) 



Figure A 


Consider Figure A which is almost identical with Figure 3 of the paper 
except for the disturbance gu (0) with 


u( 0) = 0 for t< 0 
u(0) = l for t^O 


which represent the offset from t = 0 that has to be compensated for 
by the adaptive system. The index of performance has been chosen 
as a quadratic one. 

For the moment assume 



y* = g(l- £ ->(0) 

(2) 

then 


x — Csmco x t + gs at u( 0) 

(3) 

and 


y 1 = {C sinco^ + ge at u( 0)} 2 

(4) 


One can take the Laplace transform of eqn (4); Figure B ( i ) gives 
the poles of this transformation. The zeros can be found in the known 
way taking into consideration the amplitude ratio. 

Assume the dynamics of G m (js) and ( 5 ) to be given by the pole 
zero pattern of Figure C, then the poles of signal y z are found in 
Figure B ( ii ). With the aid of the real multiplication theorem 


L [A (0 •/,(*)] = 2 ^ 


c + jfoo 


F 1 (s — v)F 2 (v)dv (5) 


c — JCO 




. /, 1 


\ L 
\ i 

k 4-Dt 


\ 


(i) 


0- 


*~2u>i 


3cj\ 

/ 

/ 

/ --2ui 


-A 


¥ 


/ \ 

\ 

\ t-2<oi 


(ii) 

Figure B 


< m )\L3 W i 


10 ] 

0 

cui 


Figure C 


479 










V. W. EVELEIGH 


it can be shown that the poles of the signal y 5 can be found by shifting 
the pole pattern of Figure B (ii) according to jco and — jco respectively 
and adding in the s plane. This leads to Figure 3 ( in ). 

The integrator adds a pole in the origin, to give the pole patterns 
of y 7 . Both poles on the real axis amount to a time function 

h(l~e~ at )u(Q) (6) 

If g = h then our initial assumption (2) for y 7 * was right. Now an 
integrator constant K can be found for which this is true. The poles 
off the real axis have been neglected in passing from y 7 to y 7 *. It is 
simple to show that the residues of these poles are at least an order 
of magnitude smaller than the residues for the poles on the axis. 

I found it very interesting to compare this treatment of the problem 
with the method of the low pass equivalent network. 

There remains one question. In experimental work I never found 
a need for an additional low pass filter G 2 ( s ); even for high speeds of 
convergence the correction signal y 7 was sufficiently smooth. So what 
is the reason for introducing G 2 (s) and accepting with it the chance of 
instability ? 
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V. W. Eveleigh, in reply 

I thank Dr. Eykhoff for his suggestion of an alternate technique and 
the example illustrating its use. Early in my study I searched for such 
an approach, but was led to the describing function method illustrated 
in the paper because of the flexibility it provides in handling various 
forms of IP surface characteristics. If the IP surface is such that the 
Laplace transform of eqn (4) in Dr. Eykhoff’s discussion can be 


readily taken, and it should often be reasonable to assume such a 
surface, then his suggested approach will work very nicely. In any 
particular situation, perhaps both approaches should be used to 
provide comparative results. 

Regarding the filter G 2 (s), I agree with Dr. Eykhoff that it is not 
generally necessary nor even desirable to use such a filter to improve 
loop performance. The primary purpose for leaving it in was for 
flexibility in gathering experimental data on loop performance. It is 
in the low pass part of the loop and adjustments there result in direct 
changes in the equivalent low pass network, thereby requiring a 
minimum of effort to determine theoretical K c , co and limit cycle 
amplitude values (if such exist). 

T. Isobe, University of Tokyo, Bunkyo - ku , Tokyo, Japan 

I highly appreciated Dr. Eveleigh’s work which treats the problem of 
extreme searching adaptive systems by deriving an equivalent low 
pass filter and enables us to get a clear image of the solution. Incident¬ 
ally a similar problem is dealt with, in the paper by T. Totani and 
myself, by using a sampled-data control system approach. I would 
like to ask Dr. Eveleigh a question. I imagine that the signal represent¬ 
ing IP or f{x) is derived from a squared error or that sort of thing. 
How may we treat its fluctuation which is caused by randomness of the 
error of control ? 

V. W. Eveleigh, in reply 

I wish to thank Professor Isobe for his comments and question and 
also point out that I found his paper very interesting. I would suggest 
that a noise analysis should be possible using additive noise at the 
non-linearity or smoothing filter output with spectral characteristics 
chosen to be representative of a specific practical situation. Simulation 
results would be readily obtained and analytical results should be 
possible to obtain using reasonable assumptions. I hope to investigate 
this problem in further detail and report on the results at a later 
conference. For the examples chosen, the presence or absence of a 
reasonable level of noise does not appreciably alter the stability 
characteristics. 
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The Realization of a Self-adapting Control 
Programme in a System with a Digital Computer 

P. F. KLUBNIKIN 


Summary 

The paper gives the results of a study of a system comprising a digital 
computer and a plant whose transfer function varies in time and is 
previously unknown. The plant and the digital computer are inter¬ 
linked via transforming units. The system uses an adaptive control 
programme by which the coefficients of the plant transfer function are 
determined (an analogue of the plant is obtained). Control of the 
plant is effected in such a way that a criterion—the mean modulus 
of the error—is minimized. Determination of the plant transfer 
function coefficients is effected during the operation of the system. 

The principle of operation of the control programme is as follows. 
In each cycle of operation of the system is performed an extrapolation of 
the input and search requirements (by use of ‘fictitious’ cycles within 
the machine) of the control signal which minimizes the system error. 

The paper gives logical schemes of the control programme, as well 
as the results of an experimental investigation. 

Sommaire 

Le rapport donne le resultat de l’etude d’un systeme comprenant un 
calculateur num<§rique dlectronique et une installation dont la fonction 
de transfert varie dans le temps et n’est pas connue k. l’avance. Cette 
installation et ce calculateur sont relies entre eux par des transmetteurs 
d’information. Ce systeme utilise un programme de commande adap- 
tative simulant le systeme au moyen duquel les coefficients de sa 
fonction de transfert sont calculds. L’adaptation automatique de ce 
systeme consisfe a minimiser le module moyen de 1’erreur de reglage. 
La determination des coefficients de la fonction de transfert du pro¬ 
cessus s’effectue lors du fonctionnement de celui-ci. 

Le principe de fonctionnement du programme de ce systeme 
d’adaptation automatique est le suivant: dans chaque cycle d’opera- 
tion du systeme, on effectue une extrapolation des valeurs d’entrde et 
du signal de commande n6cessaire k la minimisation de l’ecart de 
reglage, ceci grace k des cycles fictifs interieurs au calculateur. 

Le rapport donne le schema fonctionnel du programme de com¬ 
mande ainsi que des resultats experimental. 

Zusammenfassung 

Der Aufsatz enthalt die Untersuchungsergebnisse eines Systems, das 
einen Digitalrechner und eine unbekannte zeitveranderliche Regel- 
strecke enthalt. Die Regelstrecke und der Digitalrechner sind fiber 
Umsetzer verbunden. Durch den Systemablauf, der einer selbstein- 
stellenden Regelung entspricht, werden die Koeffizienten der Uber- 
tragungsfunktion der Regelstrecke bestimmt (man erhalt ein Analogon 
der Regelstrecke). Als Kriterium der Regelung der Anlage dient das 
Minimum des Mittelwertes fiber den Betrag des Fehlers. Die Bestim- 
mung der Koeffizienten der Ubertragungsfunktion der Regelstrecke 
geschieht wahrend des Regelablaufes des Systems. 

Das Arbeitsprinzip des Regelvorganges ist wie folgt: In jeder 
Periode T des Arbeitsablaufes des Systems wird eine Extrapolation 
durchgeffihrt, die sowohl die Forderung an die EingangsgroBe als 
auch an die Selbsteinstellung (dieRechenmaschine besitzt eine „fiktive“ 
Periode) bezogen auf das Regelsignal, das den Systemfehler zum 
Minimum macht, berficksichtigt. 

Die Arbeit enthalt Strukturschemen des Regelvorganges und 
experimentelle Untersuchungsergebnisse. 


Introduction 

Recently there has been a wide expansion in systems in which 
the control of a load is achieved using a digital computer. In 
these systems by means of the application of the proper control 
programme DCM it is possible to obtain a self-adaptive (self¬ 
organizing) property, even in the case where there is no a priori 
information on all the properties of the load and the change 
of the load parameters in the course of time. 

The elements of the theory of the construction of self- 
adaptive control systems are known, but their application 
in practice often results in very substantial difficulties connected 
with the special digital computing systems. The main difficulty 
is the determination of the characteristics of the load (for 
example, the transfer function) under conditions of normal 
operation of the system and the search for the control signal 
input to the load, which gives, from one or another point of 
view, the best control process. 

This paper is devoted to the questions of the realization of 
a self-adaptive control programme in a system with a digital 
computer. One method of constructing a self-adaptive control 
programme is considered, which permits it to be realized 
relatively simply. Results of the experimental investigation of 
control systems are presented. 

The Method of Self-adaptation in the Control Programme 

Consider an automatic control system consisting of a con¬ 
tinuous part (the load) and a digital computer (DCM, Figure 1). 
The DCM operates in a realm of periodic repetition of a 
programme with a time cycle T. 


*0 

DCM 

d 

Load 

| X 


fl 


r 








Figure 1. Diagram of the control system 


Let the link between the control input X 0 and the output 
quantity of the system X be given in the form 

X*=W s (z)X * (1) 

where X * and X 0 * are the values of X and X 0 at the moment 
of time T ; W s (z) is the transfer function of the closed-loop 
system; z = e -3 — &~ Ts is a lag operator. 

Then as is known 1 " 3 , in order for (1) to be satisfied the 
control system can be realized in the form of the block diagram 
shown in Figure 2, where for the transfer functions of the 
elements of the programme the following conditions should be 
satisfied 

D 1 ( Z )=TF s ( Z ),D 3 ( Z )=-g| (2) 
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where W H (z) is the discrete transfer function of the load. The 
transfer function D 2 (z) is chosen arbitrarily. In the most simple 
case 


D 2 (z) = 


C i z + C*2 
C 3 z +1 


(3) 


However, it is impossible to satisfy the conditions (2) and, 
consequently to obtain the prescribed properties of the system, 
when the transfer function of the load W H (z) is unknown or 
when its coefficients have an unknown time dependence, which 
is often the case in practice. It should be noted that in the 
indicated situation a general control programme, calculated in 
the presence of complete information about the load, is usually 
not convenient. 


gradient method for the magnitudes of the coefficients A i} B , 
which give a minimum of the mean difference 

1 m 

A av ~m ^ — | 

m v = 0 

where m is the number of cycles for averaging 
X* = W H (z)d* 


— Whm ( z ) d* 

Whm(z) is a ‘model 5 transfer function for the load formed 
in the DCM. This method is illustrated in Figure 3. 



Figure 2. Diagram of the control programme 


Therefore, the first step in a self-adaptive control programme 
is the determination of the discrete transfer function of the 
load, which is written in the form 



A n z n -hA n _ 1 z n 1 + ... A t z 
...B x z + B 0 


(4) 


where n is the order of the load equations; Aft), Bft) are 
time-dependent coefficients (/ = 1, 2, 3,..., nj = 0, 1, 2, n). 

Consider that the computing-time cycle of the DCM is 
chosen so that the coefficients Aft) and Bft) are unable to 
change significantly over several cycles, and that n is unknown. 
Then in order to determine during the process of operation of 
the system the current values of the coefficients At and B j9 and 
consequently W H (z) for a given moment of time, it is possible 
to use two simpler methods. 

The first method is similar to that described in a previous 
work 4 and is based on the solution of a system of equations, 
which is obtained by using the expressions (4), i. e. 


X k B' 0 + X k + 1 B' 1 +X k + 2 B' 2 + ... +X k+n B' n 

== d k+1 A[ + d k+2 A 2 + ... -\-d k+n A' n (5) 

where k = 0, 1, 2, 2 n 

d k = d(t-kT)uX k =X(t-kT) 

1 are the values of the input and output quantities of the load 
measured in the A:th preceding calculation cycle; A- and B/ 
are the approximate values of the coefficients for the current 
calculation cycle. 

/ The system of eqns (5) is solved on the DCM relative to 
A/ and B/ by one of the known methods, for example by the 
method of iterations, and Wh(z) is determined thereby. 
The second method uses the principle of a ‘learning model’ 5 
and includes the following. 

Using the values of X k and d h (k = 0, 1, 2,..., n) available 
in the memory of the DCM, a search is carried out by the 



Figure 3. Diagram illustrating the method of determining the coefficients 
w r{z), A i9 B j 


The second stage of the method described for building a 
self-adaptive control programme is the determination of a con¬ 
trol signal d 9 which will guarantee the stability of the system 
and satisfy (1) or a better approximation to this condition. As 
a criterion for the approximation to (1) it is more useful to 
select the mean absolute value or the mean square of the error 


1 * 

&av X! ^-01 

A v — 0 

s *v=i- 'Zkit-vT)] 2 
A t> = 0 

where X is the number of averaging cycles: 


( 6 ) 


S * = [W s (z)-W^ Z )lXt 


W s (z) is the transfer function of the closed-loop system. 


W r 2 \_ ( z ) [Pi ( z ) D 2 ( z ) + £>3 (z)] 

"1 +D 2 {z)W b {z) 


(7) 


Substituting in (7) the values of D x (z) and D 2 (z) from (2), 
one gets 

w h(z) , 

:+D 2 (z) W h (z) 


Ws(z) = 


W HM (z) 


1 +D 2 (z)W h (z) 


-W s (z) 


Obviously in the general case W HM (z) # W H (z) and conse¬ 
quently W s '(z) # lY s (z). However, as experimental investiga- 
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tions have shown, even a relatively rough approximation 
Whm(z) to Wh(z) for the condition of stability of the closed- 
loop circuit of the system {Figure 2), gives a behaviour of 
the system that is close to that prescribed. 

Thus on the basis of (6) and (7) one has 

£ a v = F{C ls C 2 , C 3 ,...) (8) 

The stability of the system is reached by seeking the mini¬ 
mum e av in the region of the coefficients C l9 C 2 , and C 3 . The 
search is carried out by means of extrapolation in the DCM of 
X 0 in r-conditional cycles and the calculation of s av E for these 
cycles. 



Figure 4. Graphs illustrating the method of constructing a self-adaptive 
programme 


X* 1E =W s (z)X* E) Xt E = (Xt E -Xt)D' 2 (z) 

Xt E =f^-X* E , X*=W HM (z) d| 

VV HM \ z ) 

£?2? = X2e "h ^3E (10) 

[Note: the quantity A C can be taken equal to unity.] 

where eJ$ E are the partial derivatives with respect to the coeffi¬ 
cients D 2 (z); k is the coefficient of a step in the direction of 
the reversed gradient; AC is the trial increment; Xe , X x e, X 2 e , 
X 3 e, dE are the values of the corresponding quantities in the 
conditional cycles within the DCM (the index E indicates extra¬ 
polated values of the corresponding quantities). 

(5) After an m step search the output signal from the DCM cl 
is calculated in accordance with the diagram shown in Figure 2. 

d{t)=X 2 {t)+X 3 (t) 

X 2 (t )=c 2 [X t (0 - x (0] ■+ c t [x, it - T) - x it - D] 
-C 3 X 2 (t-T) 

X J (t) = G 1 X 0 (t-T) + G 2 X 0 (t-2T) + ... + G m X 0 (t—mT) 
X 3 0 1 )(0+j-(Gi J5i + G 2 ) X 0 (t - T ) 


The idea of the method is explained in the diagrams shown 
in Figure 4. As a result, for each cycle of the DCM the following 
order of operations is obtained: 

(1) Input X 0 and X. 

(2) Extrapolation of X 0 in r-conditional cycles. 

In the simplest case for linear extrapolation from the preced¬ 
ing cycle one gets 

X 0E (t + rT) = r IX o (t) -X 0 (t- T)] +X 0 (t) (9) 

(3) Determination of the coefficients of Wh(z). 

(4) The search for the minimum e av E in the region of the 
coefficients D 2 {z ) taking into account the next r cycles. 

For the method of the modified gradient, on the basis of (8), 
one has the following formulae 

Sav£ AC 


(Ci)_ C 2 +AC, C 3 )-F(C t> C 2 , C 3 ) 

s av£ _ AC 

( c 3 ) F a (C 1 ,C 2 ,C 3 + AC)-F(C 1 ,C 2 ,C 3 ) 

e avB - AC 

AC 2 = —ks[yE 

AC 2 = — fcfiavE 

AC 3 = — keivE 

n' ( \ (C 1 +AC 1 )z+C 2 + AC 2 
u ^ z) ~ (C 3 + AC 3 )z + 1 


+ ... +-L Bn G m X 0 lt-(m+n-l)T]-^X 3 (t-T) 

-... -Ax 3 [Mn-l)T] (IT) 

A 1 


Here one takes 


W s (z) = G 1 z + G 2 z 2 + G 3 z 3 + ... +G m z m 

(6) The output of the control signal d and the updating of the 
information in memory. 

In Figure 5 is shown the logical flow diagram of a self- 
adaptive DCM programme which assures that the given opera¬ 
tions and the calculations according to the formulae (9)-(ll) 
will be carried out. Circles indicate conditional transfer opera¬ 
tors (transfer control), and the conditions are written inside 
them. As can be seen from the flow chart, 15 cycles are provided 
for in the control programme, in the course of which normal 
control is achieved according to the diagram shown in Figure 2. 
This is required for the accumulation of information in the 
DCM. No particular explanation is required for the remainder 
of the flow chart. 

It must only be noted that the number of conditional cycles 
in the DCM must be chosen so that the time for accomplishing 
the operations described does not take longer than the cycle 
time T. If, for a minimum number of conditional cycles (one or 
two), it is not possible to satisfy this condition, then it is necessary 
to use a DCM that is faster acting (in which each arithmetical 
or logical operation is executed in less time). 


483 





P. F. KLUBNIK.IN 



Figure 5. Flow chart of the control programme 
p-number (p> Ic) defining input condition of information in X k , d k 


Results of Experimental Investigations 

In carrying out the experimental investigation of the load in 
the system of Figure 1 its dynamic model was changed. The 
dynamic model of the load was linked with the DCM through 
a device transforming a voltage into an 8-digit binary code or 
the code into a voltage. The control input X 0 is supplied in the 
form of a voltage and fed through the transforming device to 
the DCM. The diagram for the realization of the control system 
during the performance of the experiment is shown in Figure 6. 

A control programme was fed into the DCM corresponding 
to the flow diagram shown in Figure 5. The dynamic model of 
the load was characterized by the transfer function 


’ S(T^S 2 +2T^S+1) (12 ' 

The quantities K, T 0 , T, and $ can be varied in time over 
the following limits: K = OT 4- 0-001; T 0 = 1-5 4 - 0-2; 7) = 
0-24-0-5; 1 = 0-2 4- 0-05. 

The rate of change of the quantities indicated did not exceed 
1—5 per cent/sec from the initial value. The calculation cycle in 
the DCM was equal to T= 0-15 sec. The connection between 
the control input X 0 and the output of the system X was given 
in the form * 

W s (z)=—(z+z 2 + z 3 ) (13) 
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Figure 6. Diagram of control system for performing experiment 
Analogue-digital transforms voltage to binary code. Digital-analogue transforms binary code to voltage 


For fixed values of the coefficients Wh(S) of (12) and with 
fulfilment of the conditions (2) the system has a first-order 
instability and a transfer process defined by (13). The rate gain 
in the system is relatively small. Its increase is limited by an 



instability in the closed-loop circuit for the selected structure 
D 2 (z) of (3). 

In Figure 7 is shown an oscillogram for the operation of 
a control system with normal control (self-adaptive programme 
excluded). As the experiment shows, for normal control the 
system is extremely sensitive to a change of the coefficients 
Z> 2 (z’), especially when this leads to an increase in the gain of the 
closed-loop circuit. In this case a change in the coefficients 
Ci, C 2 , and C 3 by 10-15 per cent makes the system unstable. 
The same effect occurs in the system with a change in Wh(z). 

By putting a self-adaptive control system into operation in 
a short time (10-15 sec) the optimal value of the coefficients 
D 2 (z) was found and the error was reduced to a minimum. In 
the process of operating, the system automatically adapted itself 
to the changed characteristics of the load. 

In Figure 8 are shown typical curves of the change of the 
coefficients IV&(z) during their determination in the DCM. The 
transfer function corresponding to (12) is written in the form 


a 

W B (z) = 


a l z + a 2 z 2 + a 3 z 3 
(1 — z) (a 4 z 2 + a 5 z +1) 


(14) 


Figure 7. Oscillogram of the evolution of an input control system with 
constant coefficients ZT (z) with normal control 
C x — 0*25, C 2 = 0*5, C 3 = — 0-25 selected by numerical means 


It can be seen from the curves that even in 6-8 sec the 
coefficients of (14) a t approximate their true values indicated 
on the graph by broken lines. 



Figure 8. Graph of the change of the coefficients a t in the process of searching in the determination of W 3M (z) 
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Figure 9. Oscillograms of the operation of the system. 


(a) portion of initial operation 

(b) portion after forming the coefficients 


Figure 10. Oscillograms of the operation of the systems: C 


(1) initial operation 

(2) after forming the coefficients 











THE REALIZATION OF A SELF-ADAPTING CONTROL PROGRAMME IN A SYSTEM WITH A DIGITAL COMPUTER 


The curves in Figure 8 were made for the very worst case, 
where the determination is carried out by a step input to the 
system applied at the time t = 0, after which X 0 remains con¬ 
stant. For an arbitrary time change of X 0 (t) the errors in the 
determination of the coefficients are significantly decreased and 
do not exceed 5-10 per cent. 

In Figures 9 and 10 are shown oscillograms showing the 
evolution of the system in the process of changing the coefficients 
D 2 (z ), C l9 C 2 , and C 3 . The oscillograms in Figure 9 correspond 
to a combination of initial values of C l9 C 2 , and C 3 for which 
the total gain of the closed-loop circuit of the system, i.e., 
Wff( 1) Z> 2 (1) is small and for which the variable input signal 
Z 0 (r) error is large. The oscillograms in Figure 10 correspond 
to initial values C l9 C 2 , and C 3 , which lead to an unstable system. 
In both cases, in a relatively small time the system auto¬ 
matically selects the optimum value of the coefficients D 2 (z) 9 
for which the error is a minimum for the given control input 
X 0 (t). 

[It is interesting to note that when the load simulator is 
switched off (.X = const.) in the course of a few cycles of the 
operation of the DCM the quantity e av E is reduced to a minimum 
and also for X 0 (t), which indicates the efficiency of the search 
method used.] 


Conclusions 

The proposed method of constructing a self-adaptive control 
programme can easily be realized in a DCM and requires a 
relatively small number of instructions in the programme. 

For the determination of the dynamic properties of the load, 
in the process of normal operation of the automatic control 
system with a DCM, it is useful to use a transfer function in the 
form Wh(z) (the equivalent of a difference equation). Here a 
good result in the determination of the coefficients of Wh(z) 
gives a method presented above, which is based on the principle 
of a ‘learning model’. 

The experimental investigation showed the efficiency of the 
self-adaptive control programme, constructed according to the 
proposed method. 
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DISCUSSION 


A. R. M. Noton, Electrical Engineering Dept., University of Notting¬ 
ham , Nottingham, England 

My question relates to the identification problem. In order to evaluate 
the coefficients of eqn (4) the author mentions two possible approaches: 
(a) that of eqn (5) and ( b ) the so-called learning model. Will he discuss 
the relative merits of the two approaches, explaining why he has 
concentrated on the second method ? 


H. H. Robertson, I.C.I. Ltd., Wilton Works, Middlesbrough, Yorkshire, 
England 

In the author’s discussion of the determination of the process transfer 
function, he considers a form given by eqn (4). This may be written 

N 


W B (z) = 


EV 

t= l _ 

i v 

i = 0 

An alternative form for this is given by 


W H (z) = 


p(z) !!(*-'«> 


where si and n are the zeros and poles respectively of the transfer 
function. It is important to choose the number of poles in the transfer 
function to correspond with the order of the system. 

The behaviour of the system can be expressed as 


Xf n = ^ qr™ + output due to input. 

i 

If the model transfer function has more poles than the actual 
system, then spurious poles have been introduced which may cause 
instability if \r\> 1. Moreover the least squares equation correspond¬ 
ing to (5) is singular and it is very likely that numerical determination 
of Ai will produce a pole with \r\ > 1. 


If no poles are taken in the model transfer function and a sufficiently 
large value of n is used, we obtain the unit impulse response function 
in sampled data form. Work on this problem has taken place at 
I.C.I. and Figure A shows a typical kernel of coefficients A%. This 
kernel was produced from data taken from an industrial plant and it 
was found necessary to pass the time series through a high-pass filter 
in order to eliminate the effects of slow drift in one of the variables. 



Figure A 


P. F. Klubnikin, in reply 

In reply to Dr. Noton, the second method of obtaining transfer 
function (4) in the digital computer was adopted because it was more 
effective. 

It was not always possible to obtain a solution of system (5) when 
the differences die... dj c +n were very small. Moreover, the solution 
required more time in computer cycles. 

The author thanks Dr. Robertson for an interesting discussion. 


487 



A Pattern Recognizing Adaptive Controller 
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Summary 


When a process is controlled by a number of parameters it may be 
possible for an optimizing controller or hill climber to seek and find 
optimum parameter values by trial and error parameter perturbation 
techniques. The general effect of uncontrolled disturbances is to 
change continually the state of the process, and the parameter values 
are required to track the changing optimum operating conditions. 
The accuracy of the tracking is limited by the relatively slow response 
of the optimizing controller since the process must be allowed time 
to respond to repeated perturbations of all the parameters. A method 
of steadily improving the tracking, and hence the average operating 
efficiency of a process, by means of a learning system has been 
developed. The learning system automatically stores information 
about the process states and the associated optimum parameter 
values. This information is utilized subsequently to recognize process 
state patterns and thereby to select the appropriate optimum para¬ 
meter values with negligible delay, since the parameter optimizer is 
no longer involved. 

The paper gives the theory of the method and shows how patterns 
that are characteristic of the process state automatically control first 
the storage and subsequently the selection of the optimum parameter 
values. A practical pattern recognizing controller (p. p. r. c.), designed 
according to the theory and employing transistor circuits, is described 
in detail. Preliminary tests, using a small number of inputs, have 
shown that the controller behaves as predicted by the theory. 

Sommaire 

Dans la commande optimalisante d’une installation a parametres mul¬ 
tiples ou les conditions optimales varient en raison des perturbations 
incontrolables, les valeurs optimales des parametres peuvent etre 
trouvees par des tatonnements successifs. La precision de Poptimali- 
seur est^ toutefois limitee par le temps necessaire aux tatonnements. 
Un systeme d apprentissage, ne pr6sentant pas cet inconvenient a ete 
d6veloppe. Ce systeme memorise automatiquement les etats de fonc- 
tionnement de l installation ainsi que les valeurs optimales associees 
de ses parametres. Ces renseignements sont ensuite utilises pour 
electionner les valeurs optimales des parametres par simple identi- 
fication de la structure de l’dtat de fonctionnement. Cette selection 
n implique pas l’utilisation de l’optimaliseur, done, elle dure tres peu 
de temps. 

On decrit la theorie de la methode et on montre comment les struc¬ 
tures caracteristiques de Petat de l’installation commandent automa¬ 
tiquement d’abord la memorisation, et ensuite, la selection des valeurs 
optimales des parametres. On ddcrit les details d’un adaptateur par 
identification de structure realise d’apres ce principe et utilisant des 
circuits & transistor. Les premiers essais, avec un nombre restreint 
d entrees, confirment l’analyse theorique. 

Zusammenfassung 

Wird eine Strecke durch eine Anzahl von Parametern gesteuert, so 
ist es emem selbstanpassenden Regler moglich, die gfinstigsten Regler- 
einstellungen durch versuchsweise Storung der Parameter zu suchen 
und zu finden. Im allgemeinen fiihren ungeregelte Storungen zu einem 
dauernden Zustandswechsel der Strecke und die Einstellwerte mussen 
den wechselnden gunstigsten Betriebsbedingungen folgen. Die Nach- 
laufgenauigkeit wird durch das relativ langsame Verhalten des an- 
passenden Reglers begrenzt, da die Strecke Zeit braucht, urn auf 
wiederholte Storung aller Kennwerte zu reagieren. Ein Verfahren zur 
stetigen Verbesserung des Nachlaufens und damit der mittleren 


Leistung der Strecke wurde aufgrund eines Lemsystems entwickelt. 
Dieses speichert selbsttatig Informationen fiber den Zustand der 
Strecke und die zugehorigen gfinstigsten Einstellwerte. Diese Infor¬ 
mation wird spater dazu benutzt, Muster des Streckenzustandes zu 
erkennen und dadurch die gfinstigsten Einstellungen mit geringster 
Verzogerung zu wahlen, weil ja ein Optimiergerat nicht mehr be- 
notigt wird. 

Der Beitrag gibt die Theorie der Methode an und zeigt, wie die 
Muster des Streckenzustandes zuerst die Speicherung und dann die 
Auswahl der gunstigsten Einstellwerte regeln. Eine technische Aus- 
fuhrung des mustererkennenden anpassenden Reglers, der gemaG der 
hier entwickelten Theorie gebaut wurde und Transistorschaltkreise 
benutzt, wird in Einzelheiten beschrieben. 

Vorlaufige Ergebnisse mit einer geringen Anzahl von Eingangen 
zeigen gute Ubereinstimmung mit der Theorie. 


Introduction 

The optimization of a complex process by means of optimizing 
controllers employing trial and error parameter perturbation 
techniques has a time course that is long compared with the 
process response time, since it must in general be based on many 
small parameter changes and the process must be allowed time 
to respond to each change. This may be the best that can be 
achieved when the parameter optimizing controller is first used, 
but if appropriate information about the state of the process and 
the optimum parameter values is selected and automatically 
stored in a permanent but adaptive memory system it is possible 
for the average optimization response time to be decreased as 
the store of information grows. Thus, if the state of the process 
can be adequately defined by taking a sufficiently large number 
of measurements, including the optimum parameter values, when 
an optimum state is finally reached, it is only necessary to 
recognize this state immediately whenever it occurs on future 
occasions and to use this information to select the associated 
optimum parameter values at the same instant. By this means 
the long initial optimization delay in response to a change in the 
process state, which may be over an hour for some processes, 
can be completely eliminated for the particular state that can be 
recognized and also for process states that so resemble it that 
they are similarly classified by the pattern recognizer. 

In practice the state of a process and the corresponding 
optimum parameters are continually changing due to uncon¬ 
trolled variables and 100 measurements may be required for an 
adequate specification of the state. For a 10 per cent accuracy 
of measurement the theoretical number of possible distinct 
states is 10 100 and in general each may require a different set of 
parameter values for optimum performance. In practice, however, 
there are usually severe physical constraints on the process states 
and on the factors that cause the state to change so that there 
may be a possibility of obtaining worthwhile increases of 
efficiency with a pattern recognition system of reasonable 
memory capacity. 
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Operating Principle of the Pattern Recognizing Adaptive Controller 

In Figure 1 the inputs to the pattern recognizing adaptive 
controller (p.r.a.c.) derived from the controlled process consist 
of N measurements i ± ... ijsr, and to illustrate the operation of 
the controller N will be assumed to be 100, as in the experimental 
system that has been constructed. The measurements may, for 
example, indicate the distribution of temperature throughout 
a vessel and in this case there will be a physical restriction on 
the temperature gradient and typical set of readings may be as 
shown in Figure 2 (a), curve A. This is in fact a sample of band 
limited random noise obtained from a noise generator and could 
represent deviations of the measurements from reference levels. 
If the measurements are not sufficient to describe the state of the 
process adequately, they may be extended to include the dis¬ 
tributions of other variables such as pressure and specific 
gravity. 


It is assumed that two parameters P x and P 2 , each having ten 
possible values, are controlled and that an optimum setting of 
these exists for the process state corresponding to the 100 
measurements with the envelope A in Figure 2 (a). At time t x the 
optimum values of the two parameters denoted by P 10 (ti) and 
P 20 (fj), will correspond to a maximum of the measure of per¬ 
formance such as M 2 in Figure 2 (b) and, if M 2 remains stationary 
or moves sufficiently slowly, Pj and P 2 will eventually reach 
their optimum values under the control of the parameter 
optimizer. Any disturbances that are sufficient to change the 
optimum operating point to M 4 at t 2i for which the optimum 
parameter values are P 10 (/ 2 ) and P 20 (t 2 ), can also be expected 
to change the state of the process and hence the distribution of 
measurements as illustrated by curve B in Figure 2(a). For 
sudden disturbances the curve will change from A to B at a rate 
determined by the process time constants, whereas the para¬ 
meters will change at a much slower rate due to the optimizer 



Figure 1 



Figure 2(a) 
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Figure 2(c) 


time constants. This optimizer lag is entirely eliminated when 
the p.r.a.c. is able to recognize the measurement distributions 
and select the optimum parameter values instantaneously on the 
basis of past information. The accumulation of this information 
will generally be a very slow process but it may be speeded up 
by introducing artificial disturbances so that a wide range of 
sample process states are covered in the shortest time that will 
allow the optimizer to track the moving optimum point. 

The principal problems considered in this paper are the 
theory of operation and design of a p.r.a.c. that will learn to 
recognize the measurement pattern and to supply the most 
probable optimum parameter values on the basis of earlier 
experience. This result could clearly be achieved in principle by 
storing the information in a digital computer of sufficient storage 
capacity but the method described here leads to a low cost, 
on-line p.r.a.c. employing a parallel memory system giving 
negligible access time to the entire memory and extreme 
reliability. The controller is thus suitable for high speed processes 
for which a digital computer would be far too slow. 


Detection of Non-linear Changes in the Measurement Pattern 

The first stage in the p.r.a.c. simplifies the input measure¬ 
ments 4 ... ijsr by forming a new set of variables h-.-jN defined by 


j r = K 


h 2 + g T h~q) 


( 1 ) 


in which the constant and linear changes in the original envelope 
are essentially eliminated. This operation is performed in unit H 
0 Figure 3) by inverting the i r and forming the appropriate com¬ 
binations in resistor networks. The j r corresponding to curves A 
and B in Figure 2 (a\ are shown respectively at C and D for the 
case when q = 3 and K = 3 in eqn (1). 

In some processes it may be necessary to take measurements 
over a two or three-dimensional field and in these cases eqn (1) 
is extended to include signals surrounding each i r , A two- 
dimensional H unit has been constructed for the experimental 
p.r.a.c. and this operates on 100 measurements taken over 
a 10 x 10 plane of transducers arranged in a hexagonal pattern 
as shown in Figure 2 (c ). The equation corresponding to this 
arrangement is 

( 2 ) 


Jr = K 


i __ _L s * x measurements 

r 6 °n circle surrounding i r ) 


The effect of the H unit in one or more dimensions can best be 
illustrated by the curves C and D of the one-dimensional 
example which tend to have peaks in the regions where the slope 
of the original curves change. Each output j r of the H unit is 
supplied to an I unit and also to an A t and B 1 unit. The latter 
replace the j signals that are above or below the average level 
by constant amplitude signals Obi and Og 2 respectively. The Ob\ 
signals for the two measurement patterns are shown at E and F 
in Figure 2 (a). 

The next sections in the p.r.a.c. learn associations between 
the j patterns and the optimum parameter values on a maximum 
likelihood basis so that the parameters selected by any new 
j pattern are initially the same as those selected by the nearest 
learnt pattern, being the most probable values. If, however, 
a new pattern is associated with different parameters during the 
learning process, then these parameters become the most 
probable values for the new pattern which becomes one of the 
learnt patterns. In other words, the p.r.a.c. makes the best guess 
on the evidence currently available and the high speed at which 
this is done is due to the parallel method of storing this evidence 
and of obtaining simultaneous overall access to it, as described 
in the next section. 


Theoretical Basis of the Learning and Recognition Process 

The N outputs j\ ...jjg of the H unit are supplied through 
diodes to the potentiometers of the information storage units /, 
as shown in Figures 3 and 4(d) , the diodes suppressing negative 
values. All the potentiometer sliders are initially at the lower 
end so that the voltages at the sliders are initially one half of the 
positive inputs or zero in the case of negative inputs. The sliders 
may be driven independently by the ratchet impulse motors in 
the direction that increases the gain ‘a’ of the I unit from the 
initial value \ towards a final value unity that is not normally 
reached. The I units are grouped first into pairs and then into M 
sections of N pairs, M being the number of quantized levels for 
each parameter. The outputs of the right-hand / units, -j T 
in each pair, are passed through a sign changing summing 
amplifier A 2 that forms 

1 N 

~at jr 




( 3 ) 
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Figure 3 


This quantity is supplied to the summing integrator D together 
with the left-hand I unit outputs a r + vV that are not changed 
in sign. The gain of the summing integrator is adjusted so that 
its output becomes 

- | Cj. + C 2 X (a r+ - a r _) j r d t (4) 

JoL r=l J 

where C x and C 2 are constants and t = 0 is the time at which 
a reset pulse, that has reset the integrator output to zero, is 
removed. The gains a r + and a r - are intially \ throughout the 
p.r.a.c. I units and the integrator outputs increase linearly in 
magnitude after a reset pulse until one of the B 2 units is triggered. 
Theoretically, the B 2 unit that triggers first is initially selected 
at random, but in practice it is found that small errors in com¬ 
ponent values cause a bias in favour of certain units. 

The general case in which the j r take on all possible values 
cannot be handled analytically and it is necessary to simplify 
the analysis by considering that all positive j r have the same 
amplitude. Let the number of positive j r be n and let all the a h- 
in the left-hand I units that receive a positive signal from a 
particular measurement pattern be changed to 1/2 + djln , all 
other a r + remaining at Conversely, let all a r ~ in the right-hand 
units that do not receive a positive signal from the same pattern 
be changed to 1/2 + djln, all other a r - remaining at Eqn (4) 
for this particular pattern and set of a r changes becomes 




4G-44)# 

p*r n d 

= - c 1 + c 2 ^j,d 

JoL r=i zn j 


since the negatives j r from n + 1 < r < N are eliminated, 
Eqn (5) assumes that the signals are rearranged in order to 
separate positive and negative values into two groups from 
r = 1 to n and r =* n + 1 to iV respectively. The magnitude of 
the changes in the a r for any given value of n is controlled by the 
sensitivity constant d and since 7? m i n = 1 and a rm&x = 1, d must 
be less than or equal to unity. It will be observed that if similar 
changes are made in the sets of a r + and a T - values that were 
unchanged in the above operation then the resultant effect is 
zero and eqn (5) becomes the integral of C x once more, since 


— c 1 +c 2 i X 


- /1 , A-l-A \ • 

^V2 + 2n 2 2 nJ Jr 
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Trigger Circuit "B # 
(b) 



(e) 




Figure 4(a-g) 



Electronic switch equivalent 
Drive Pulse Generator" C 


This facility can be used to cancel the effect of any changes that 
are subsequently found to be in error or inefficient as, for 
example, when the parameters supplied initially correspond to 
a secondary lower maximum such as M 1 or M s in Figure 2 (b). 

The next step in the theory will be to show that any chosen 
pattern of n (1 < n < N) equal non-zero signals of any amplitude 
and N — n zero signals will cause the B% unit supplied by 
/units with n corresponding gains a r at 1/2 + dj2n and (N - n ) 
corresponding a r - gains at 1/2 + dUn to trigger before any 
other B 2 unit, after a reset pulse. The closest possible sets of 
/ unit gains to the particular set chosen are the two sets that 
have only one difference, namely the set with (it - 1) gains 
flr+ at 1/2 + d/2n and (N - n + 1) gains at 1/2 + d/2n and 
the set with (n + 1) gains n r+ at 1/2 + d/ln and (N — n — 1) 
gains a r - at 1/2 + d!2n. Let the contributions of these two sets 


to the integrands be q n - x and q n+1 respectively and let the con¬ 
tribution due to the chosen pattern be q n . When the chosen 
pattern is present at the p.r.a.c. input terminals the value of q n , 
as given in eqn (5), is 



dj 

2 


( 6 ) 


since the n — non-zero j r are assumed to be equal. The values 
°f Qn+ i and q n - x for the same input pattern are 
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and since 


_ffl, d l\. 

q n+i + 2(n + l) 2/ Jr 

n dj 
n + 1 2 

(n-2) n_ 

(n-1) n + 1 


( 8 ) 

( 9 ) 


it follows that the input to the B 2 unit whose I units ‘match 5 the 
input pattern will rise more rapidly than the inputs to all other 
B 2 units and will therefore reach the trigger level first. The 
common feedback circuit from the B 2 unit through the diodes K 
to all integrators acts as a holding device and prevents any 
other B 2 unit from triggering. Thus only one of the M outputs 
On — O x m is at the triggered level at any instant. Each output 
is supplied at an appropriate gain, e.g. 0 1X x 1 unit, 0 12 x 2 
units, etc., to the summing amplifier E [Figures 3 and 4 (e)] to 
form the quantized value of the first parameter p{. The M 
quantized levels may be given any desired spacing distribution 
by adjusting the amplifier input resistors R n — R x m [Figure 
4(e)l 

It has been stated that one of the important characteristics 
of the p.r.a.c. is that a pattern not previously encountered 
automatically produces the same parameter values as the 
pattern that it most closely resembles in all the patterns that have 
been presented in the past. This can easily be seen from eqn (9) 
which shows that the integrand produced by the two patterns 
with one more or one less signal than the pattern that has 
adjusted a set of I units is nearly the same as the input due to 
the latter and will therefore trigger the same B 2 unit. This small 
pattern difference can, however, cause different B 2 units to 
trigger when the optimum parameter values are different, so 
that special sets of I units are formed to resolve the pattern 
difference. 

Having established that a certain distribution of a r + and 
a r ~ values is required to recognize a pattern, it remains to be 


shown that the distribution can be established automatically 
by the pattern itself. The actual set of 2N /units that is adjusted 
to match the presented pattern must also be selected so that 
the correct optimum parameter value is produced when the 
pattern is automatically recognized after it has been presented 
once during the learning process. This selection is controlled 
by the optimum parameter value for the process state. This 
value, denoted by p 1 for the first parameter, is supplied to the 
F unit (. Figure 3 ) which quantizes it into M levels and indicates 
the level by triggering one of the M outputs Qi ~ Qm* These M 
levels are made to correspond with the E unit outputs and since 
the Q output at any instant causes a B 2 unit to trigger, the value 
of Pi is always a quantized representation of p x when the 
switch S x is closed during the learning process. No change takes 
place in / units until the performance indicator sends a negative 
gate signal G to the C unit when the optimum or desired per¬ 
formance is attained [Figures 3 and 4 (c)]. In the case of a hill¬ 
climbing parameter optimizer the G signal will indicate that 
the peak has been reached. 

Each C unit has four binary inputs Ob x , Ob 2 , G and B, for 
which a T is represented by -20 V and a 4 0 5 by zero volts 
approximately. The output Ob x from each B 1 unit (. Figure 3) 
is 4 1 5 if its input signal j is above the average j and the output Ob 2 
is T if its input is below the average. Each C unit has two 
outputs r and r which have a resting potential of + 20 V that 
prevents the / units being driven. When G is T and j > j or 
when B is T and j < j the output r becomes — 20 V which 
allows the / units supplied by it to be driven by the drive pulses 
so that the appropriate a h-, in the set of I units released by a 
B 2 unit output, are allowed to increase. The second output r 
becomes -20 V when G is T and j <j or when B is T and 
j > j thus allowing the appropriate a r - to increase. 

Signal B is used to cancel the effect of G if an error is made, 
or it may be used in trial and error learning to eliminate all 
unsuccessful trial parameter values so that by a process of 
elimination the optimum values slowly become more probable 
and are finally selected. 



Figure 5 (a) 
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Figure 5 (b) 


The magnitude of the change in gain in the sets of I units 
corresponding to the optimum parameter values is required 
to be proportional to \jn. This is achieved by driving the units 
for a fixed time T with pulses of a frequency that is proportional 
to 1 /n, supplied by the drive pulse generator L. A simplified 
equivalent circuit of the L unit is shown in Figure 4 (g). In 
practice an electronic switch is used to generate a repetitive 
sequence of pulses of height proportional to 1/n (1 < n < N) 
and these are added to a voltage proportional to n, derived from 
the B 1 units. The resultant waveform is used to trigger a circuit 
similar to the B x unit and the output drive pulse frequency is 
thus proportional to 1/n. The drive pulses supply all the / units. 


and when both diodes in a unit are non-conducting the ratchet 
motors turn the potentiometer sliders [Figures 4 (d), 5 (d) and 
5(b)] which increase the gain 4 a ’ by d/2n in time AT. The sen¬ 
sitivity factor d can be adjusted to match the time constants of 
the process by selecting appropriate values of the maximum 
possible drive pulse frequency, the gear ratio and A T, but once d 
is chosen it must remain constant during the learning process. 


Termination of the Learning Process 

Throughout the learning process the switches S t in Figure 1 
are closed to supply the optimum parameter values to the 
p.r.a.c. As the learning becomes effective the outputs of the 
p.r.a.c. labelled p{ ... p Jc f become the same as p x ... p k for an 
increasing range of process states, but the learning process is 
only terminated when a criterion is reached. This criterion 
might, for example, be reached when ninety out of the last 
hundred sets of different parameter values are predicted 
correctly by the p.r.a.c. When the criterion is satisfied switch S 2 
may be changed over to connect the p.r.a.c. to the process and 
switch S 1 opened to terminate the learning process. When the 
p.r.a.c. is in control it is still possible for new states of the 
process, requiring parameter values that have not been supplied 
during the learning process, to occur. The parameter values 
supplied by the p.r.a.c. will then be those that were the optimum 
values for the past process state that most closely resembles the 
existing state and it is improbable that they will be very far 
from the true optimum values. 

Preliminary tests, using a small number of inputs, have 
shown that the p.r.a.c. behaves as predicted by the theory. 
Experiments with patterns consisting of 100 measurements are 
in preparation and the results will be reported as they become 
available. Several thousand I units will eventually be available 
for storage and Figure 5 (b) shows two of the standard panels, 
each containing 200 I units. 
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DISCUSSION 


P. H. Hammond, National Physical Laboratory, Teddington, Middlesex , 
England 

The author’s proposals are based on a steady-state open-loop learning 
phase, during which a given state variable pattern q ... i N is recognized 
as requiring given steady-state parameter settings p 1 ...pj c to establish 
optimum steady-state operating conditions. The subsequent operating 
phase uses the pattern-recognizing adaptive controller (p.r.a.c.) as an 
element in a closed feedback loop which includes the process. An 
immediate difficulty arises here. For a state variable pattern which 
happens to be present the p.r.a.c. gives as an output the best parameter 
setting based on experience in the learning phase. The process then 
responds to the changed parameter settings by producing a changing- 
state variable pattern. This, in turn, modifies the parameter settings 
and it is not at all obvious that the system will converge to a stable 
state. Even if it is possible to achieve stable operation in the absence 
of detailed knowledge of the process dynamics, the time to settle 
may be considerably larger than the process time constants. 


The techniques discussed in the paper are certainly applicable when 
the inputs i to the p.r.a.c. are quite independent of the process para¬ 
meter settings, i.e. process inputs and disturbances. The p.r.a.c. is then 
not part of a feedback loop but rather a feedforward controller. 

The scheme uses a very large number of / units. It would seem that 
this number could be reduced by seeking efficient transformations of 
the state variable pattern, e.g. in terms of polynomials of which the 
coefficients provided inputs to the I units. 

There are two small points on which I would welcome the author’s 
comments. They are: 

(1) Why is it preferable to take the second differences of the /’s 
to form the fs rather than the first differences? 

(2) Why is it necessary to determine the mean value of the fs 
separately in unit A rather than using the sign of j itself to drive the 
B x units? 
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W. K. Taylor, in reply 

I agree with Mr. Hammond that it is not obvious that a system con¬ 
trolled by the p.r.a.c. will converge to a steady state, and more work 
is required on this problem. A steady state is not an essential condition 
for optimum operation, however, since the optimum parameter 
settings are chosen for each state. In many applications the state 
would, in any event, be continuously changed by unknown disturb¬ 
ances which may have a stronger influence than parameter changes. 

It may be possible to reduce the number of I units by employing 
additional transformation of the state variable pattern as Mr. Ham¬ 
mond suggests, but only at the expense of introducing complex 
equipment between the process and the p.r.a.c. On the question of 
whether to use the first or second difference, the second difference 
was chosen because it gives transformed patterns that have less overlap 
at the I units, thereby facilitating their independent classification. If 
linear changes are of particular importance it may be useful to employ 
additional H units, taking first as well as second differences. 

The difference between j and j was taken to eliminate the effect 
of signal components that may be common to all H unit outputs and 
to suppress the random effects of low amplitude noise voltages. 


A. A. Feldbaum, Institute of Automatics and Telemechanics , Kalan- 

chevskaya 15a , Moscow , U.S.S.R. 

The direction of a design of such self-adjusting systems which utilized 
the combination of principles of automatic search and pattern recog¬ 
nition will, it seems to me, become of utmost importance in the near 
future. The very interesting work of the author is a first step in this 
direction and it is natural that many important questions are not 
considered in this paper. 

The choice of the form of connection between function para¬ 
meters (being extremized) and pattern values is of great importance. 
Instead of establishing a direct relation between extremum coordi¬ 
nates and regions of pattern space, it seems more convenient to pre¬ 
assign these coordinates as some functions of plant input para¬ 
meters and time. The unknown coefficients in the formula for these 
functions can be placed in correspondence with certain regions of 
the pattern space. 

One of the basic questions in systems with the combination of 
automatic search and pattern recognition is the relation between these 
two methods with the choice of the control action (input parameter 
of the plant). It is assumed in the paper that the recognizing device 
has first completely learned and then replaced a search system. 
Evidently a system as a whole will perform more efficiently if the 
recognizing device will participate in control before accomplishing 
the learning process simultaneously with the search system. Moreover, 
in the course of a learning process the degree of participation of the 
search part will decrease. 

W. K. Taylor, in reply 

Professor Feldbaum also asks about the possibility of forming some 
functions of the state variable pattern. It may be possible to obtain 
an advantage by introducing this complexity in some applications, 
but in others it is possible that the original patterns are in the best form 
for supplying to the p.r.a.c. The time element can be introduced into 
the system as it stands by supplying delayed patterns as inputs in 
addition to the direct inputs. 

The suggestion that both the search and learning system could 
be allowed to operate simultaneously is quite feasible. In the extreme 
case the search system or hill climber can be dispensed with and a 
trial-and-error type of learning used with the p.r.a.c. This has been 
investigated on a smaller scale but has the disadvantage that the 
probability of selecting optimum parameters at random is initially 
very small unless there is a coarse quantization of parameter values to 
start with, the quantization becoming finer as learning proceeds. 


O. L. R. Jacobs, Department of Electrical Engineering University of 

Edinburgh , Edinburgh 9 , Scotland 

I comment on the time scales involved in the described system. 

The system in Figure 1 of the paper is controlled in the first 
instance by an extremum-seeking parameter optimizer. This can only 
be effective if the disturbances that cause changes of state and of 
optimum parameter values occur slowly compared with the extremum 
search time. On the other hand, the p.r.a.c. is designed to recognize 
changes of state in a time short compared with the extremum search 
time. It seems then that the system to be controlled is one in which 
disturbances arise as sudden changes at infrequent intervals. 

Could the author please give examples of systems where disturb¬ 
ances occur in this manner, and where there is no other way of 
recognizing that a disturbance has occurred than by using the p.r.a.c. ? 

W. K. Taylor, in reply 

The disturbances may arise at relatively frequent intervals after the 
p.r.a.c. has taken over control, as compared with during the learning 
process. Examples of systems where disturbances may occur suddenly 
would be a process in which a raw material suddenly changes in 
characteristics, or a moving vehicle that suddenly enters a region of 
changed physical conditions. A digital computer could be used to 
recognize the changing states, but for complex patterns it would be 
many orders of magnitude slower than the p.r.a.c. which is a direct- 
coupled, parallel-acting network. 

D. A. Bell, A.M.F. British Research Laboratory, Reading , Berks, 

England 

As I am not experienced in process control I have been unable to 
visualize a system which has only two controllable parameters, but 
from the measurement viewpoint appears to have 100 degrees of 
freedom. Could Dr. Taylor give an example? If, as in his first example, 
the measurements relate to temperature distribution, surely the occur¬ 
rence of local hot spots would require local remedial action rather than 
adjustment of the two overall parameters? 

The essence of Dr. Taylor’s case is that the internal relationships 
which reduce the apparent 100 degrees of freedom to little more than 
two are so complex that they cannot be explicitly stated; hence his 
illustrative curves in Figure 2 {a) have been constructed as samples 
of random noise. Is there not a danger that the device will store patterns 
which are actually random noise, and will give only a random rela¬ 
tionship between ‘pattern’ and optimum values of the two parameters? 

I should have thought that in many cases the nature of the pattern 
would be apparent—e. g. temperature gradient in a distillation column 
—and the specific characteristics of the pattern could be directly 
abstracted and used to bias the direction of search of the optimum¬ 
seeking controller. Dr. Taylor’s detection of non-linearities, a process 
equivalent to that known as ‘edging’ in visual pattern recognition, is 
an important step, but he still assumes an apparently random distribu¬ 
tion of edges. Is this likely in practice? 

In conclusion, I would like to make it clear that I am not attacking 
learning machines as such, but I suggest that for industrial processes 
they should be avoided as far as possible. To use an animal analogy, 
for a comparatively narrow range of goals and environments involved 
in any particular industrial process it should be possible to use instinct 
(i.e. built-in behaviour patterns) rather than learning. 

W. K. Taylor, in reply 

I would like to emphasize that the p.r.a.c. is a perfectly general system 
that is easily adapted to any number of inputs and outputs. The 
example of 100 inputs was chosen to illustrate the possible complexity 
of a process state pattern. The system could easily be taught to 
recognize local hot spots and to take appropriate action when they 
arrive. The state patterns of a process may easily look like samples 
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of random noise but there are usually many physical constraints that 
limit the range of possible states. I agree that there are many simple 
cases in which the patterns of interest are fixed and known a priori. 
The controller could then be used with fixed potential dividers provid¬ 
ing appropriate predetermined weights in place of the / units together 
with the A 2 , D, B 2 and E units. This combination would perform 
what Dr. Bell calls direct abstraction of specified characteristics and 
would be a pattern-recognizing controller without the adaptive 
elements. 


Y. Sawaragi, Kyoto University , Kyoto, Japan 

I think that eqn (4) plays the most important role in this paper and 
that the controller which you stated is constructed on the basis of this 
equation. I would like to ask about its mathematical foundation from 
the viewpoint of data processing. 


W. K. Taylor, in reply 

I should have mentioned that eqn (4) is not the only function on which 
the design of the p.r.a.c. may be based. It has been found in experience 
to be a convenient function of the inputs but there is some reason to 
believe that non-linear functions of the inputs would give superior 
pattern discrimination. It is doubtful whether the gain would be worth¬ 
while, however, considering the technical difficulties. 


There is only one point cpi e Ft for which the following conditions 
are satisfied , „ . 

(/;> (pi)>(fj,(pd r (1) 

where ft e Ft and fj e Fj 
Transform (1) to the form: 


(/;, <pd>c- t where c ; =min(/;,<■/>;) B (2) 

Let the points (p it corresponding to (ft, cpi) > max a be labelled as 
generalized images, and the points a as thresholds of identification. 
It is quite clear that sets of points <pt and c% will fully characterize a 
system of automaton images. 

The points 9 h and C{ can be easily found by using the algorithm 
that was worked out at the Institute of Automatics and Telemechanics, 
U.S.S.R. [Automatics and Telemechanics , No. 5 (1963)] 

The images of the automaton proposed by Dr. Taylor can be treat¬ 
ed as well as the points pi, but the difference is that these points 
are determined after showing a single pattern which belongs to a 
particular image, and the method I am speaking about provides 
determination of all points in question by successive showing of 
patterns belonging to different images. Those patterns which are 
identified during the learning process (determination of points q>% and 
Ci) will be taken with a zero weight, and those that are not will be used 
to correct the corresponding coordinates of points <pi and a. 

Therefore the last method provides a higher degree of accuracy, 
and in addition it can be used in the case when fi is expressed in any 
desirable form. 


W. K. Taylor, in reply 


V. Sokolov, Institute for Automatics and Telemechanics , Kalanchevs- 

kaya St. 15, Moscow, U.S.S.R , 

The paper presented by Dr. Taylor is very interesting as a whole. 
Nevertheless I would like to make some comments on that part of it 
which is concerned with a problem of pattern recognition. I hope that 
the approach which I am going to propose now is more common and 
would provide wider possibilities. 

Let us assume that we have a set of plant patterns with correspond¬ 
ing points, say cp, in the receptor space and some single meaning trans¬ 
fer functions, say X, chosen from a given set of transfer functions U 
(X e U). The transfer function X will determine for every point fe cp 
one point in a functional space. (In the most general case it will be 
Hilbert’s space.) We say that a set of patterns cp is divided into n 
images if it is possible to divide a set of points cp into n subsets <Pi ... cp n 
in such a way that the corresponding points in Hilbert’s space will 
have the following properties. 


I agree that it is possible to obtain greater accuracy in pattern recog¬ 
nition by choosing more complex functions of the state variables that 
in effect discard less information about the process state than do the 
functions chosen in the paper. The question that should be asked, 
however, is whether the resultant increased cost and complexity of 
the controller is justified by the improvement in control. The answer 
will depend upon the particular application and may only be ob¬ 
tained after detailed experimental investigation. 

In reply to Mr. Sokolov’s remarks on the training procedure, I 
would like to emphasize that it is, in general, necessary to show pat¬ 
terns belonging to different ‘images’ and the two methods would ap¬ 
pear to be equivalent. Thus, although a large set of related patterns is 
given the same classification by presenting a single member, it is pos¬ 
sible to subdivide the set by successive presentations of members that 
are required to produce different outputs. 
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STELLA: A Scheme for a Learning Machine* 

J. H. ANDREAE 


Summary 

A scheme for a learning machine is described. In a basic exploratory 
mode the machine searches its environment. Learning enables it to 
profit from those action sequences which lead to reward. By correlating 
the changes in its environment with its actions, the machine can 
extract invariant features and use them to guess profitable actions when 
its learned sequences fail. An internal mode of operation is described 
in which the machine explores the possibilities of its future actions 
with a view to modifying its performance. The learning automaton, 
STELLA, is being constructed in the form of a mechanical tortoise 
which takes its name from its laboratory origin. 

Sommaire 

On decrit le schema d’une machine a apprentissage. Dans un mode 
fondamental d’exploration la machine cherche son amb'ance. 
L’apprentissage la rend capable de profiter de celles des sequences 
d’action qui conduisent a la recompense. En correlant les variations 
de l’ambiance avec ses actions, la machine peut extraire les traits 
invariants et les utiliser pour deviner les actions profitables quand ses 
sequences apprises lui font defaut. Un mode interne de fonctionne- 
ment est decrit dans lequel la machine explore les possibility de ses 
actions futures en vue de modifier ses performances. L’automate a 
apprentissage, STELLA, est en cours de construction sous la forme 
d’une tortue mdcanique qui tient son nom du laboratoire d’origine: 
Standard Telecommunication Laboratories Ltd. 

Zusammenfassung 

Der Aufsatz behandelt die systematische Anordnung eines lernenden 
Automaten. Durch einen grundlegenden Suchvorgang „erforscht“ die 
Maschine ihre Umgebung. Lemen befahigt sie, diejenigen unter den 
moglichen Handlungsfolgen (Schritten) auszusuchen, welche zu einer 
„Belohnung“ fiihren. Durch Korrelation der Veranderungen in ihrer 
Umgebung mit ihren Schritten kann die Maschine nicht veranderliche 
Merkmale feststellen und gewinnbringende Schritte „erraten“, wenn 
keine der gelernten Schritte anwendbar sind. Ein interner Arbeits- 
vorgang wird beschrieben, mit welchem die Maschine die Moglich- 
keiten von zukiinftigen Schritten im Hinblick auf Abwandlungen ihrer 
Arbeitsweise erforscht. Die Konstruktion des lernenden Automaten 
hat die Form einer mechanischen Schildkrote; der Name STELLA 
kommt von „Standard Telecommunication Laboratories 46 , wo er ge- 
baut wurde. 


Introduction 

STELLA is a scheme for a general-purpose learning machine. 
For convenience, the scheme is being tested in the form of a 
mechanical ‘tortoise 5 moving about on a level floor which can 
observe the approximate distances and angles of obstructing 
walls. The scheme is intended to be unspecific to the particular 
type of environment in which the machine operates, and other 
machines and environments are under consideration. 

* Progress made with the STELLA learning machine since this 
account was submitted on 23 rd August 1962 is the subject of a paper 
being prepared for publication. 


There would appear to be two main approaches to the design 
of learning machines. In the one, a rigid control system per¬ 
forming a strictly determined task is modified to make it less 
rigid, more flexible and adaptive. In the other, a general-purpose 
learning machine is designed and is then modified to make it 
more specific to particular problems. STELLA falls within 
this latter category and the object of experimenting with the 
STELLA scheme is to explore ways in which it can evolve into 
a more specialized system. The importance of this process of 
‘progressive segregation 5 of parts is emphasized by Bertalanffy 1 . 

STELLA, the tortoise, is a self-propelling and self-steering 
trolley which can move about on a level floor. Walls of standard 
height having illuminated upper rims obstruct its movements 
in various directions. Information about the angular positions 
and distances of these walls is obtained by photocells mounted 
around the circumference of the trolley. There are eight of these 
photocells on the machine which is being constructed. Figure 1 
assumes six only. 

The Exploratory Mode 

Unless its memories are ‘primed 5 by the experimenter, 
STELLA starts with no information about its environment. In 
these circumstances, and whenever the higher level of operation 
fails to decide its actions, STELLA proceeds in an exploratory 
or search mode. This mode is governed by a random generator 
in such a way that the machine tends to perform different actions 
when receiving the same information repeatedly from the 
photocells of its eye. 

Each photocell of the eye has a fixed threshold which the 
light input to the photocell must exceed for the information 
from that photocell to change from — 1 to +1. Thus each photo¬ 
cell contributes one ‘bit 5 to the input binary pattern from the 
eye—the eye pattern. 

The actions of STELLA are restricted, in Figure 1 and in the 
constructed machine, to four movements: forward, forward to 
the left, forward to the right, and reverse. 

The short-term memory shown in Figure 1 consists of a 
6 x4 matrix of binary elements represented by the intersections 
of the six column lines from the eye and the four row lines to 
the dissimilarity detector (jDZ>). Each row line is associated with 
one of the output movements, forward, left, right or reverse. 

When the eye ‘sees 5 a pattern, the DD selects that row in the 
short-term memory along which is stored the pattern least 
similar to the eye pattern. The motor output control then per¬ 
forms the output movement or action corresponding to the 
selected memory row. The continuous random improver of the 
memory switches, by random choice, some of those elements in 
the selected row which are not the same as the corresponding 
elements of the eye pattern, until, due either to the movement 
causing a change in the environment or to the random improve¬ 
ment of the row, that row ceases to be the least similar to the 
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eye pattern and the DD selects a different row. The selection 
of least similar rows follows the procedures of Steinbuch 2 ; 
the random improvement is superimposed to provide a 
short-term memory which eliminates repetitive reactions to the 
eye pattern: STELLA should not push persistently against 
walls and should break out of repetitive cycles. 

Learning by Reward 

Patterns and actions can be stored in the two parts of the 
long-term step memory shown in Figure 1. At the same time, 
and in the same row, a connection can be made in the sequence 
store to indicate that the pattern and action of that row was 
followed by a pattern and action stored in another row, or by 
reward. Each intersection between a row line and column line 
in the sequence store represents the following of the action of 
that row by the pattern in the row connected to that column line; 
or, if it is the reward column (marked ‘pleasure’ in Figure 1 ), the 
following of the action of that row by reward. There may be 
different kinds of reward with separate columns, and there may 
be a ‘pain’ column; these are considered in later sections. 

When an eye pattern is received, it is compared with the 
patterns stored in the long-term step memory by the threshold 
detector ( TD ). The TD associates any row of the memory which 
has a pattern more similar to the eye pattern than the current 
threshold of similarity demands. Consider three cases: 

(a) The TD associates no row, but the action prescribed by 
the DD leads to reward. The eye pattern and the subsequent 
action are stored in a vacant row of the long-term step memory 
and a connection is made in the sequence store between that 
row and the reward column. The row is given unit weight in the 
weight store. 

(b) The TD associates one row. The TD overrides the DD 
and causes the motor output control to perform the action 
stored in the associated row of the long-term step memory. At 
the same time, suppose that there is a connection in the sequence 
store which indicates that on a previous occasion the action 
was followed by the pattern in another row of the long-term 
step memory. The weight of this other row is increased in 
anticipation of its being seen again. Also, while the action is being 
performed, the preceding pattern and action are held tentatively 
in a vacant row of the long-term step memory; in fact, every 
pattern and action is held tentatively for one cycle. Now, the 
action performed, several situations may ensue: 

(/) If there is no association of the new eye pattern by the TD 
and no reward, the weight of the anticipated row is reduced to 
what it was, the tentatively held pattern and action are dis¬ 
regarded, the weight of the last row to be associated by the TD 
is reduced, and the machine STELLA reverts to the search mode. 

(ii) If reward is received, the weight of the associated row is 
increased, a connection in the sequence store is made to the 
reward column, and the tentatively held pattern and action are 
stored with unit weight in a row with a sequence connection to 
indicate the row which followed. 

(iii) If the anticipated row is associated by the TD , its weight 
is increased further in confirmation, the tentatively held pattern 
and action are held for another cycle and the next step is entered. 

(iv) If an unanticipated row is associated by the TD , the 
weight of the anticipated row is decreased and the weight of the 
associated row is increased. 


(i?) If the TD associates more than one row, one has, as for (c). 

(c) The TD associates more than one row. Each of the 
associated rows has a weight depending upon how many times 
it has been used in the past and each row is connected to the 
reward column through a series of connections in the sequence 
store. Each connection in this ‘circuit’ of the sequence store 
between the row and the reward column represents a possible 
step. If each connection is imagined to have an electrical 
resistance and the column and row lines of the sequence store 
are imagined to be wires, then the total resistance between an 
associated row and the reward column is a measure of the number 
of steps expected to lead to reward. The random row selector 
chooses one of the rows associated by the TD on a chance basis 
biased towards high row weights and short paths to reward. 
Once a single row has been chosen, the situation is the same 
as for (b). 

Figure 2 is a flow diagram of the processes outlined above, 
together with other processes which will be described. 

The threshold of the TD determines the permitted latitude 
in the association of row patterns with the eye pattern. If the 
threshold of similarity is low, the behaviour will tend to be 
illogical and opportunity will exist for the discovery of shorter 
paths to reward. When STELLA is being particularly successful 
in obtaining rewards, it is necessary for the threshold to be kept 
high so that previously established paths will be followed more 
precisely the more successful they prove to be. However, it is 
not desirable that a pattern should be completely disregarded in 
favour of search behaviour because the threshold happens to be 
just too high. Therefore, it is proposed to have a slow decline 
in the value of the threshold with time, to cause the threshold 
to be restored to a maximum when reward is received, and to 
allow the TD to depress its own threshold (temporarily) by not 
more than a predetermined amount below its current value, if no 
pattern is associated. 

‘Forgetting’ is necessary to remove those steps which are 
less used and, therefore, less useful so as to make room for more 
important steps. To do this the weights of the rows in the long¬ 
term step memory are given slow time decays. When a weight 
falls below some prescribed value, the contents of the row are 
erased and all connections to and from the row in the sequence 
store are broken. 

The ‘teacher’, shown in Figure 1, is a control which overrides 
the motor output control to enable the experimenter to speed up 
the learning process by forcing STELLA along profitable paths. 

Pain 

There will be some values of the parameters controlled by a 
learning system which must be avoided. For example, in the 
control of a chemical plant it may be known that certain mixtures 
of reactants are explosive and that these must be avoided; again, 
in a traffic control system it is likely that there will be a minimum 
safe distance of approach of vehicles. In the tortoise floor-wall 
system one can imagine holes in the floor which the tortoise must 
avoid in order to survive. Clearly it is inadequate to provide 
learning by reward only in a machine which can act by trial and 
error in its environment if forbidden conditions are to be 
avoided. It must be taught or be programmed to avoid these 
conditions and the information which determines this avoidance 
will be called pain. Responses to pain cannot be maintained by 
experience in the way responses to reward are reinforced. 
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Figure 2 


A traffic control system could not be allowed to maintain a safe 
distance of approach between vehicles by regular experiments 
with unsafe distances. 

Pain is introduced into the STELLA scheme of Figure 1 by 
the addition of a ‘pain 5 column to the sequence store and by an 
externally controlled ‘pain warning 5 . In order to understand the 
implication of the negative sign attached to the pain column, 
which may be compared with the positive sign ascribed to the 
pleasure, or reward column, it is convenient to think in terms 
of the actual electrical method employed for the operation of the 
random row selector. In order to bias the decision of this 
selector according to the length of the path (number of steps) 
to reward, the connections in the sequence store are resistors and 
the row line in question is earthed, while the reward column line 


is connected to a positive battery voltage. In this way a current 
flows from battery positive to the earthed row which is inversely 
proportional in magnitude to the resistance of the path, that is, 
the length of the path. Now some connections to the pain column 
will be preprogrammed into the machine and some will be formed 
as a result of the pain warning. If the pain column is connected 
to a negative battery voltage, the currents to this column, in so 
far as they pass through the same resistors as the current from 
the positive reward column, will oppose the currents from the 
pleasure column and decrease the probability that the random 
row selector will choose the row in question. Actions expected 
to lead to pain will be inhibited according to how many steps are 
expected before pain may be encountered. If the pain currents 
exceed the positive currents, it is arranged for the action pre- 
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scribed by the row to be blocked. That action is forbidden for 
that step. 

The connections to the pain column are made in the same 
way as for those to the reward column, but allowance is made for 
preprogrammed connections to the pain column to be established 
for some rows with permanent (non-decaying) weight. 

Specialization 

In order that STELLA should learn paths out of painful 
situations, the cessation of the pain warning should be treated 
as a reward and an additional positive reward column is required 
tor the sequence store. This column would become connected 
to the positive battery voltage when a pain warning was received 
and it is logical to arrange for the other reward column to be 
disconnected simultaneously. Suppose that the machine is given 
a goal and the pleasure reward indicates the achievement of this 
goal. If the implication of a pain warning is sufficiently serious, 
as it is intended to be, then the final attainment of the goal is 
subject to a temporary diversion to the subsidiary goal of 
escaping from the painful situation. Similarly, if the system has 
to ensure the supply of its own power or the supply of raw 
materials to the process it is controlling, it may be essential for 
this secondary goal to be attended to at the expense of progress 
towards the prime goal. This logical interdependence of the 
operation of the reward columns forces upon us the specializa¬ 
tion of the system to match its environment. The machine must 
be told, to begin with, what is required of it. It must also be 
given some rules to prevent it from destroying itself or others. 
Natural selection is too costly. 

What other kinds of specialization have to be introduced? 
The scheme of Figure 1 was designed to be unspecific to its 
environment. The connections from the eye can be interchanged 
so that different photocells control different columns of the 
memories and after a time the system should adapt itself to the 
new arrangement. The same applies to the connections to the 
output actions. But this interchangeability of connections 
presumes that each photocell is providing equivalent information; 
otherwise patterns stored in the long-term step memory will 
give undue importance to digits on some of the columns at the 
expense of those on others. 

It seems possible to arrange for STELLA to compensate for 
the unequal importance of digits in the eye pattern. Each 
column is given a variable weight which determines the bias 
applied to the appropriate element of a pattern associated by the 
DD or TD. The column weight is decreased every time the TD 
disregards the corresponding digit of the eye pattern in associat¬ 
ing a pattern in the long-term step memory with the eye pattern. 
After a time some columns will be contributing little to the 
operation of the machine and the experimenter can change the 
position or other property of the respective photocells in order 
to achieve an arrangement in which these photocells contribute 
their full share. The machine could, of course, be programmed 
to make such changes itself. However, the experimenter might 
find that more radical changes were needed. For example, it 
might be more effective to start with 100 photocells logically 
connected to give only the six outputs, these outputs representing 
more specific characteiistics of the environment. See, for example, 
Lettvin et al? 

The variation in the column weights should represent the 
relative importance of the information arriving on the various 


columns, when the importance is averaged over a large number 
of steps, and it should indicate the efficiency of the photocells as 
receptors of information. The relative importance of elements of 
a pattern stored in the long-term step memory will depend, not 
only on the efficiency of the receptors, but also on the situations 
in which the pattern is used. Suppose that the experimenter 
decides to reward STELLA (the tortoise) every time it reverses 
on approaching a wall. As things have been described, STELLA 
would have to store a number of patterns representing the various 
situations in which it approaches a wall, each of these patterns 
being coupled with the action ‘reverse’. If the machine is not 
being very successful in obtaining reward and the threshold is 
low, then it may be lucky enough to use one of these patterns in 
a situation which does not quite correspond to the remembered 
situation, but this is not possible when it is more successful. 
Now let each element of each pattern in the long-term step 
memory have variable weight, the weights for each pattern being 
normalized so that the TD will still just associate this pattern 
with the eye pattern if it is identical and if the threshold is at 
a maximum. The element weights are modified each time the 
row is selected by the random row selector, disregarded elements 
having their weights lowered. The experiment envisaged above 
will lead to the establishment of a pattern in the long-term step 
memory coupled to the action ‘reverse’ with its element weights 
adjusted so that only those digits of the pattern, which indicate 
the presence of a wall ahead, contribute to the association 
procedure of the TD. A crude kind of generalization takes place 
so that a number of possible patterns representing a particular 
situation are accepted and remembered as a single pattern. 

The Correlation Level 

It was stated above that to begin with the machine must be 
told what is required of it, but this is not strictly true. It can 
explore its surroundings and learn something about their 
characteristics while the experimenter is still making up his mind 
about what he wants the machine to do. STELLA does this 
constantly by means of its correlation matrices, which are shown, 
drawn one upon the other, in Figure L Each of the four matrices 
is associated with one of the actions. 

Every eye pattern received is applied to the columns of the 
matrix appropriate to the action which has just been performed. 
The same pattern is then held for one step (one action) and 
applied to the rows of the matrix corresponding to that action. 
Thus, for any action, the appropriate matrix has the initial 
pattern applied to its rows and the resultant pattern applied to 
its columns. So long as the action has not been prevented by the 
external environment, each element of the matrix has its ‘value’ 
increased (decreased) if its row and column have the same 
(opposite) binary values (+ 1). 

When STELLA has explored the environment for some time, 
the correlation matrices should contain experience of the way 
in which its actions transform the observed environment. 
Invariants in these transformations will be reinforced by per¬ 
sistent increases or decreases in the values of particular elements 
of the matrices. For example, in the tortoise floor-wall arrange¬ 
ment one would expect the matrix corresponding to the forward 
movement to contain the information that an observed pattern 
will move past the tortoise from front to back. 

The correlation matrices are used if the TD fails to associate 
a pattern when a remembered path is being followed. The 
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sequence of operations is shown in Figure 2. The unassociated 
pattern is applied to each of the correlation matrices in turn 
until the TD associates one of the transformed patterns. If the 
TD associates one of the transformed patterns, then the action 
corresponding to the matrix which effected the transforma¬ 
tion is performed, as a hopeful guess based on experience. The 
transformation by one of the matrices is carried out by matrix 
multiplication of the pattern binary vector by the non-binary 
matrix, the signs of the components of the product vector 
determining the binary elements of the transformed pattern. 

If the guesses resulting from the correlation matrices some¬ 
times enable STELLA to rediscover the paths which it loses, they 
will contribute to the efficiency and speed of learning. There is, 
however, a more significant way in which the correlation 
matrices can take part in predictive forecasts. This is the subject 
of the next section. 


The Internal Mode 

If the reception of eye patterns and the performance of 
actions are blocked, an internal mode of operation can be 
envisaged which might be called ‘dreaming’. The last eye pattern 
to get through before the blockage leads by the DD or the TD 
to the selection of an action but, instead of the action being 
performed, the corresponding correlation matrix is used to 


transform the eye pattern into a ‘guessed’ second pattern. This 
second pattern is now treated as a new eye pattern, the DD and 
TD select a correlation matrix to form a third pattern, and so 
on. The ‘dreams’ may lead sometimes through the random excur¬ 
sions of the DD and sometimes more logically by the TD 
through the remembered paths of the long-term step memory 
to occasions of pain and pleasure. 

In the internal mode of operation, STELLA is exploring the 
possibilities of future actions by using the information stored 
in the correlation matrices and in the long-term step memory 
to anticipate the effects of its postulated actions. If time is 
allotted during the performance of each action (see Figure 2) for 
short excursions in the internal mode, and if some variation of 
the weights of rows in the long-term step memory is permitted 
when the excursions anticipate reward or pain, then STELLA 
can modify its own actions according to the machine’s pre¬ 
dictions. 
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Automatic Control Learning Systems 
(in the Light of Experiments on Teaching the Systems 

Pattern Recognition) 
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Summary 

The paper is devoted to the problem of teaching automatic systems 
pattern recognition. At the basis of the experiments that were con¬ 
ducted with regard to teaching, lies a profound and original hypothesis 
relating to the teaching process—a so-called ‘compactness hypothesis'. 
This hypothesis is interesting in that it opens a path to the mathemati¬ 
cal description of the essence involved in the teaching process, which 
description is not attached to any concrete technical realization of the 
learning system nor to any concrete pattern type. At the same time, it 
prompted the experiments described in the present paper, and also 
made it possible to comprehend already-known studies on teaching. 

The fact that the experiments on teaching described in this paper 
were conducted on a universal (multi-purpose) computer is naturally 
tied in with the presence of such an hypothesis. The ‘compactness 
hypothesis’ makes it possible to formulate various teaching algorithms. 
Two of them (the algorithm of random planes and the algorithm of 
potential surfaces) were verified experimentally and yielded good 
results. 

Sommaire 

Cc rapport traite lc probleme de Fenseignement de la reconnaissance 
automatique de la structure des systemes. Une hypothese profonde 
et originale, concernant le processus de Fenseignement, et dite ‘hypo¬ 
these de compacitd’ a <§t6 dmise et a servi de base d’experimentations. 
Cette hypothec ouvre la voie a une description mathematique de 
Fcsscnce memo du processus d’enseignement, independemment de 
toutc realisation concrete de systeme d’apprentissage et du type de 
structure concrete & reconnaitre. Elle a permis les experiences decrites 
et a permis de mieux comprendre les etudes connues sur la meme 
question. 

Les experiences ddcrites dans ce rapport ont £te effectuees sur un 
calculateur universel. L’hypothese de compacite a rendu possible la 
formulation d’algorithmes d’enseignement. Deux d’entre eux (Falgo- 
rithme de plans al&itoires, et Falgor thme de surfaces potentielles) 
dtaient vdrifi<5s experimentalement et avaient donne de bons resultats. 

Zusammenfassung 

Der Aufsatz befaBt sich mit dem Problem, automatischen Systemen 
die Zeichenerkennung zu lehren. Die Versuche dariiber bauen auf 
einer tiefschiirfenden neuartigen Hypothese des Lernvorganges auf, 
der sogenannten „compactness hypothesis 44 (umfassende Hypothese). 
Diese Hypothese ist deshalb interessant, weil sich durch sie das 
Wesen des Lernvorganges mathematisch beschreiben laBt; diese 
Beschreibung ist weder an eine bestimmte technische Verwirklichung 
des Lernsystems noch an eine bestimmte Zeichentype gebunden. Sie 
war auch der Ausgangspunkt fur die hier beschriebenen Versuche und 
ermbglichte die Einbeziehung bereits bekannter Untersuchungen fiber 
das Lernen. 

Die Tatsache, daB die hier vorgelegten Lernversuche auf einer 
Universal-(Mehrzweck-)Maschine durchgefuhrt wurden, hangt na- 
tiirlich auch mit dieser Hypothese zusammen. Die „compactness 


hypothesis 44 ermoglicht es, verschiedene Lernalgorithmen zu formu- 
lieren, von denen zwei, der Algorithmus der zufalligen Ebenen und 
der der Potentialfelder, mit guten Ergebnissen geprfift wurden. 


Introduction and Setting up of the Problem 

The term Teaming automaton’ has varying interpretations, 
depending on whether the existing automatic control systems 
or automatic control systems of the near future are in mind. 

In learning control systems, which are beginning to be 
employed more and more widely in present-day techniques, by 
teaching, one generally means the process involving the gradual 
improvement of the system’s operation under definite opera¬ 
tional conditions, or the capacity of the system for varying the 
adjustment changing the operational conditions. Such instruction 
is accomplished through the action of ‘reward’ and ‘penalty’ 
signals, which evaluate the results of the control system and are 
employed in the system for the purpose of seeking optimum 
alignment. These signals are sent out by an operator or by 
another automaton, but in any case the goal of the instruction 
is the alignment of dynamic system characteristics. 

The successful solution, by present-day techniques, of prob¬ 
lems of this type, lays the basis for expecting, in the near future, 
a considerably more intensive and delicate utilization of the 
teaching process in automatic control systems. 

The possibilities involved in teaching living beings, even 
including those that stand on the lowest rungs of the evolu¬ 
tionary ladder (for example, worms), are still incommensurably 
greater than the possibilities of teaching the most advanced 
automatic control systems. How far is it possible to advance in 
imitation of the learning process of the living brain by means 
of the automatic machine? In the near future, may a material 
expansion be expected in the possibilities of teaching automatic 
control systems? Many expressions of opinion have been 
published, in recent years, containing general points of view on 
questions of this nature, but, unfortunately, throughout the 
entire world, very few experiments have been conducted and 
published, that would be in a position to give substantiated 
answers to such questions and offer a glimpse into future 
techniques that could be used in teaching automatic control 
systems. 

Even though, in the setting up of such experiments, the very 
broadest understanding of the term ‘teaching’ is kept in mind 
(teaching accommodation to changing conditions, teaching the 
most favourable behaviour, and so forth), the experiments are 
usually conducted under conditions of teaching the automatic 
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machines the recognition of patterns. The capability of the living 
brain for learning pattern recognition is one of its most delicate 
capacities, and any advance in imitation, by the automatic 
machine, of this feature of the living brain’s activity immediately 
indicates the paths for the reproduction, by the automatic 
machines, of other, simpler learning processes, too. 

The present report is devoted to a consideration of this study, 
which was carried out in the laboratory (under the supervision 
of the author) of the Institute of Automation and Telemechanics, 
by Braverman 1 , with the participation of O. A. Bashkirov and 
I. B. Muchnik, and is tied in with the series of problems solved 
in the papers of Rozenblatt 2 , Selfridge 3 , Steinbuch 4 , Bongard 5 
and others. The studies with which this paper is concerned differ 
from those of References 2, 3 and 4, with regard to the ideas 
on which the teaching is based, and also, because Rozenblatt, 
Selfridge and Steinbuch constructed, for the purpose of imitating 
the teaching process, specialized automatic machines (Rozen- 
blatt’s Perceptron, Selfridge’s ‘Pandemonium’, Steinbuch’s learn¬ 
ing matrix, etc.). On the other hand, this paper’s experiments 
were conducted on universal electronic machines. 

When speaking of teaching the recognition of visible patterns, 
the following two differing processes are in mind: 

(1) The instructor knows more than the student. The 
teacher is not only capable of recognizing the patterns, but has 
also reasoned them out in advance, as must be done to conform 
with each concrete, partial case. Knowing in advance which 
visible patterns the pupil will have to recognize, the instructor 
explains to the pupil how it must be done, i. e., he assigns a 
programme, so to speak—the recognition algorithm. The pupil 
memorizes this programme and carries out the recognition of 
the visible patterns afterwards shown to him in conformity 
with it. Such understanding of the term ‘teaching’ does not 
indicate any interest for the purposes of automatic control. This 
understanding brings one only to the usual programming prob¬ 
lems which always arise in the theory of digital computers. 

(2) There is yet another teaching process, another capability 
of the living brain, that is not understood up to the present time. 
If a pupil, not knowing the alphabet, is shown the various 
outlines of several alternatives of the letters ‘a’ and ‘b’, and is 
told that this letter is ‘a’ and that letter ‘b’ then, after a certain 
period of time, the pupil will be able to distinguish the letters ‘a’ 
from the letters ‘b’ and, at the same time, distinguish not only 
those that were shown to him previously, but also all the 
remaining outlines of these letters. 

An automatic imitation of precisely such a teaching process 
constitutes our purpose, if teaching in control systems is borne 
in mind. 

Up to this point, in speaking of a ‘visible pattern’, the 
author has depended merely on the intuition of his audience. 
The concept is now established more accurately. 

By a visible pattern, is meant an infinite set of visible per¬ 
ceptions; for example, plane figures, representations, which is 
remarkable with regard to the following: people, seeing only 
some elements of this set, confidently attribute to it other 
elements not previously seen. 

Thus, for example, a set of portraits of all the persons in 
attendance here, does not constitute a pattern because if a 
person not present here were shown some of these portraits, 
he would not be able, afterwards, on seeing a new portrait, to tell 
us whether the face represented on it was that of a participant 
at our gathering. The concept of a ‘male portrait’, however, does 


constitute a pattern, because, even though no one has seen all 
the male portraits, each one of us, upon seeing a new portrait, 
would be able to say, with a high degree of assurance, whether 
it represents a male portrait. Other examples of patterns are 
represented by the concepts of the ‘number two’, the ‘letter a’, 
a ‘circle’, a ‘landscape’, etc. 

When in teaching a machine the recognition of visible patterns 
is spoken of, the following experiment is in mind. Imagine that 
there is a photo-field made up of a large number of photo¬ 
elements. Images are projected on to the photo-field—the 
elements of some kind of patterns, for example, some sort of 
outlines of the numbers two and three. In each projection, from 
the output of each photo-cell, there is taken off a current of 
definite intensity, i.e., the entire photo-field reaches a certain 
state. From each photo-cell, wires are led up to the device that 
we call an automaton. 

In the teaching process, from the entire uncountable (non- 
enumerable) amount of pattern elements, for example, of the 
outlines of numbers 2 and 3, a few, say 20, are selected and 
shown to the machine by projecting them on to the photo-field. 
In each display, it is communicated that by pressing a definite 
button, a two, a three, and so forth have been demonstrated. 
After all the selected numbers have been shown, the teaching 
process is concluded, and the pushbutton keys are switched 
off (cancelled). 

Following this, on the photo-field there are projected various 
other outlines of twos and threes that had not been used pre¬ 
viously. The machine, by some sort of conventional signal, for 
example by switching on an appropriate panel indicator, gives 
an answer to the question: ‘What is this?’ One will be able to 
say that a teaching process has been produced if the machine, 
in an overwhelming majority of cases, gives the correct replies. 
The requirement is that, without changing anything in the 
machine circuit or in its programme, the same test with new 
patterns can be repeated. For example, it should be possible, 
without changing anything in its programme, to take a machine 
that has been taught the recognition of numbers, and by means 
of the same process, reteach it the recognition of letters or 
geometric figures, or teach it to distinguish male and female 
portraits, and so forth. 

Compactness Hypothesis 

By the very nature of the problem, it is not known in advance 
which concrete patterns the automatic machine will have to 
learn to recognize. For this reason, a set of signs cannot be 
included in the programme that is suitable for the recognition 
of a given concrete pattern. Only two paths which could lead 
to the solution of the problem are seen: 

(1) The machine programme would contain so many of the 
most varied signs, which have either been fed into it previously 
or developed by the machine itself, that this set of signs would 
be sufficient for the recognition of various patterns. 

(2) There should be found and fed into the programme some 
kind of single, or small number of universal, ‘signs’, that lie at 
the basis of the very concept of ‘pattern’, and which are not 
specific for the recognition of concrete patterns. 

It would seem that such a universal sign exists, that it is 
closely associated with the above-mentioned concept of‘pattern’, 
and that it may be formulated in the form of a hypothesis, which 
was called a ‘compactness hypothesis’. 
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In Figure 1 , is the representation of several patterns for the 
outlines of numbers five and three. If these pattern elements 
were shown to a large number of people, one by one, and they 
were asked: ‘What number is this?’, then the decision would be 
practically unanimous: in the upper row, fives are represented and 
in the lower one, threes. With reference to the numbers represent¬ 
ed in the centre row, it is certain that there would be no una¬ 
nimity. In going from one ‘five’ to another, all the intermediate 
pattern elements will also be perceived by the majority of viewers 
as fives, and in a smooth transition from a five to a three, there 
will necessarily appear pattern elements with regard to which 
opinions will be divided, and it will no longer be possible to as¬ 
semble an overwhelming number of voiced opinions towards the 
clarification of what number is being represented. In addition 
to this, if the representation of the figure five (or three), is altered 
to a negligible degree in any direction, then the new pattern ele¬ 
ment will likewise be perceived as the number five (or three). 

55 5 5 
53 33 
3 3 3 3 

Figure 1 

The compactness hypothesis assumes that all the patterns, 
in fact, possess these properties; namely (a) that a smooth 
transition is always possible from one element of a given pattern 
to another; this takes place in such a way that all the intermediate 
pattern elements will be perceived as elements of this pattern. 
On the other hand, it is not possible to have a smooth transition 
from the elements of one pattern to the elements of another 
without the development, ‘on the way’, of pattern elements, 
regarding which opinions would be divided, and, without having 
a lack of unanimity with respect to which one—the first or the 
second pattern—to relate them, and (b) the pattern limits are 
not exceeded in the case of a minor pattern element alteration 
in any direction. 

Let an w-dimensional space be matched up with a photo¬ 
field containing ‘n’ photo-cells in such a way that along each 
coordinate axis there is laid off the condition of one of the 
photo-cells. In that case, a point in this space corresponds to 
each photo-field condition, and will be called a receptor space. 

Now project some type of pattern element on to the photo¬ 
field. All the photo-cells will assume some condition, that is, 
a definite photo-field state will correspond to this pattern ele¬ 
ment, which means that a single point in the receptor space will 
be obtained. 

The compactness hypothesis confirms that a domain in the 
receptor space corresponds to all the pattern elements that com¬ 
prise some type of pattern, and that different domains corre¬ 
spond to different pattern elements, without having any com¬ 
mon points. 

Keeping this ^-dimensional pattern in mind, it will be 


represented conditionally on a two-dimensional plane. In that 
case, it can be visualized that two separate domains exist 
objectively in that plane—one of them corresponds to the first 
pattern, for example to the figure ‘five’, and the second one to 
the second pattern, for example to the figure ‘three’. The domain 
boundaries are not known in advance. 

In the teaching process, a certain amount of random- 
selected elements of the first and second pattern is shown, i.e., 
a certain amount of random points, respectively, from the first 
and second domains. The problem of the automatic machine 
consists in that, knowing only these points, it should be able to 
lay out the surface of separation in such a way that the domains 
appear distributed along various sides of this surface. 

When, at the end of the teaching process, there will appear new 
pattern elements or, what amounts to the same thing, new points 
in the receptor space, the machine will be able to answer the 
question as to what patterns these points belong to, dependent 
only on what side of the surface of separation these points lie. 

If the surface of separation is successfully drawn, and the 
domains are completely divided by it, then the machine’s 
replies will always be correct, no matter how many new pattern 
elements belonging to these patterns were shown. 

The aim, at the present time, consists in finding automatic 
algorithms for laying out the surface of separation, proceeding 
only from a small number of random selected points, from 
domains whose boundaries are unknown. 

Teaching Process Algorithms 

Two automatic algorithms were suggested for the purpose 
of solving the problem regarding the construction of a surface 
of separation for emerging random points, in the case where the 
boundaries which underlie the domain division, are unknown. 

First algorithm—algorithm of Fandom planes'. Assume that 
at first, in the teaching process, two points appeared, and that 
the machine was informed that they corresponded to different 
patterns (for example, the figures three and five were shown). 
In that case, the machine traces a plane that is selected at 
random, with a single, limiting condition that it must separate 
the emergent points [. Figure 2 (a)]. If, subsequently, a point 
appears once more, and it turns out that two points, which 
belong to varying patterns, are not divided by a plane, then a 
new, random plane is drawn that divides them [Figure 2 (b)], and 
so on, during the entire teaching process. In this manner, at the 
end of the teaching process, a large number of traced random 
planes [Figure 2 (c)] have been stored in the machine’s memory. 
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They break up the entire space into domains, in which any 
(and all) like points that have been shown are distributed, or 
else there is not a single indicated point at all. Following this, 
and in accordance with a predetermined algorithm, sections of 
the planes are ‘erased’ from the machine’s memory, on both 
sides of which sections like points are distributed, or else those 
plane sections are erased in which points, shown during the 
teaching process, are distributed on one side, while on the other 
side there are shown no points at all. As a result, a sectionally 
broken hypersurface is formed, which is made up of plane 
sections [Figure 2 (d)]. On different sides of this surface, points 
that belong to different patterns are distributed. The surface thus 
plotted is taken as the dividing one, and after that, upon being 
shown a new pattern element, i.e., upon the appearance of 
new points, the machine answers the question, ‘What is this?’ 
depending on which side of the surface the point lies. 

In view of the fact that the planes are traced haphazardly 
each time, the repetition of the experiment will lead to the 
construction, not of the same, but of another surface of separa¬ 
tion. This makes it possible to ‘form parallel variants’, and as 
a result of this, to increase the probability, to a large extent, of 
accurate replies, if during the teaching process there are con¬ 
structed in parallel several surfaces of separation, and following 
hat, the problem is allowed to be solved ‘in accordance with the 
majority of votes’. 

Second algorithm-algorithm for construction of potential 
surfaces. Upon being shown some point or other, the machine 
constructs a function of the point of receptor space, which attains 
a maximum in the point shown, and, remaining positive every- 
where, decreases on being withdrawn from it in any direction. 
When several like points are shown, i.e., points that belong to 
one domain, the machine also constructs functions for each 
of these points, and afterwards combines them. It can easily be 
seen, that if the machine is shown several points that are distrib¬ 
uted in a relatively uniform fashion through the domain, then, 


jFtx^Xa) 



as a result, the machine constructs a surface, a distinctive 
potential function, which has high values (‘hump’) in domain 
points and drops sharply on being withdrawn from it. If two 
domains have to be separated, for example, then the machine 
constructs two such functions from the indicated points of these 
domains; one of them has a ‘hump’ over the first domain, and 
the second one over the second domain (Figure 3). In the case 
of the second algorithm, the teaching process is concluded with 
the construction of this potential function. Subsequently, upon 
being shown new points, the machine solves the question as to 


which domain these points belong, in accordance with which 
one of the plotted potential functions has a greater value 
at this point. In the case of the second algorithm, the surface of 
separation is represented by the projection of the line of inter¬ 
section of the plotted potential surfaces. 

The author, together with Mr. Rozonoer and Mr. Braverman, 
proved some theorems connected with algorithm convergence! 
It was proved that using this algorithm in a finite number of 
steps it is possible to separate two domains if these are ex¬ 
pressed in some simple conditions such as are always satisfied 
m practical problems. Furthermore the algorithm convergence 
will exist in spite of how one chooses the potential function as 
long as the function satisfies some simple conditions. Moreover, 
it is not necessary for the function to have the maximum in the 
shown point, to remain positive everywhere and to decrease 
on being withdrawn from shown point in any direction. These 
three conditions are more specific than necessary and are 
mentioned only to explain the algorithm more simply. 

Experiments on Teaching the Machine 

In order to verify the compactness hypothesis and that of 
the two described algorithms, experiments were conducted on 
universal digital computers by E. M. Braverman (for the first 
algorithm) and O. A. Bashkirov (for the second algorithm). 

In the first series of experiments (on the first algorithm), 
160 outlines were prepared for each of the five figures—0, 1, 2, 3 
and 5, i.e., 800 figure specimens altogether. For the purpose 
of instructional material, 40 specimens of each figure were 
selected, i.e., 200 figures, and 600 figures specimens were 
employed for subsequent machine testing. The results of these 
experiments are shown in the first col umn 0 f Table 1. 

It is evident, from the table, that in each variant the number 
of correct machine replies reached 89 per cent, and ‘paralleling’ 
seven variants, the number of correct replies comprised 98 per 
cent. Under these circumstances, it turned out, that at the end 
of the teaching process, there were taken up from 1,500 to 
3,000 binary digit bits by the machine’s memory. 

In the second series of experiments (in reference to the 
algorithm for plotting potential functions), the operation was 
carried out both with five, as well as with all ten, figures simul¬ 
taneously (see Table 1 , fourth and fifth columns). In this case, 
four times less material was made use of for teaching purposes, 
i.e., for teaching purposes only 10 specimens of each figure were 
selected. On dividing five figures, in accordance with this algo- 
rithm, the machine yielded 100 per cent correct replies to 750 
questions (using 150 specimens of each figure). During the ex¬ 
perimental period, while teaching the machine simultaneous 
recognition of all 10 figures, 85 per cent correct replies were 
obtained to 1,400 inquiries. There is a clear understanding as to 
how to improve the algorithms, so that in this case, too, the 
number of correct replies might approach 100 per cent. 

Experiments are being conducted, at the present time, on the 
utilization of the described algorithms for the purpose of teach¬ 
ing the machine the recognition of more complicated patterns; 
for example teaching the machine the simultaneous recognition 
of all 33 letters of the Russian alphabet in manuscript outline, 
the recognition of letters picked up by various typographic 
characters in the presence of ‘obstacles’ (smears, blurs, hap¬ 
hazard transposition of letters, etc.), the recognition of male and 
female portraits, images of the ‘en face’ and profile type, and so 
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forth. Despite the fact that these experiments have not yet been 
concluded, the successful results of the experiments with figures 
reassures us and gives us hope, that in more complex cases, too, 
it will be possible to obtain acceptable results. 


Table 1 




First 

algorithm 

Second algorithm 



First 

experiment 

Second 

experiment 

Dividing 


Figures 

Figures 

Figures 

Patterns 


0, 1, 2, 3, 5 

0, 1, 2, 3, 5 

0,1,2, 3, 4, 
5, 6, 7, 8, 9 

Pattern 





elements 

Of each figure 

160 

160 

150 

Prepared 

Total number 

800 

800 

1500 

Selected for 

Of each figure 

40 

10 

10 

Instruction 

Total number 

200 

50 

100 

Used for 

Of each figure 

120 

150 

140 

Verification 

Percentage 
of Correct 

Total number 

600 

in one exper¬ 
iment, 83-89 

750 

1400 

Replies 


On ‘parallel¬ 
ing 7 seven 
variants, 

98*5 

100 

85 


Observations 

First Observation. Along with the concept ‘pattern 7 , the con¬ 
cept ‘association of patterns’ can also be introduced. For ex¬ 
ample, the concepts ‘small letter “a ” 7 and ‘capital letter “A 7 ” 
are patterns, whereas the concepts ‘any “a 77 7 and “‘A 77 letter 7 
do not constitute a pattern. In fact, if a child, who does not 
know how to read, was shown only a row of outlines of the 
letter ‘small “a 77 7 , and told what it was, and was then shown the 
letter ‘capital ; ‘A 77 7 , and asked ‘What is this? 7 , then naturally, 
he would not give the correct reply. In order to teach the child 
(by the second teaching method) to recognize any letters, he 
must be shown ‘small “a 77 s 7 as well as ‘capital “A”s 7 . In 
such cases, it is convenient to speak of ‘association of patterns 7 . 

Both described algorithms are suitable for teaching the 
machine the recognition of pattern associations, if, in the teach¬ 
ing process, there are shown (approximately uniformly), the 
points of all the associated domains. Thus, for example, in 
utilizing the potential surfaces method, a ‘two-hump 7 surface, 
rather than a ‘one-hump 7 one will now be associated with the 
joining of the patterns ‘any letter “a 77 and “A 77 7 . 

In fundamentally the same manner, it is possible to teach the 
machine to distinguish between any rectangles and triangles and 
any circles and ovals; between any figures and letters; between 
the portraits of three given faces and set of portraits of five 
other given faces, and so forth. 

Second Observation. The described algorithms, in principle, 
always divide the patterns, but, of course, are not convenient in 
all cases, inasmuch as in some cases, for the purpose of division, 
too long a display may be required in the teaching process. The 
method is suitable in those cases where the domains, which 


correspond to the patterns, are distributed in the receptor space 
at a good distance from one another. If, however, the patterns 
are such that the surface of separation must be ‘strongly bent 7 , 
then teaching by the described method will require the display 
of a very large number of points. 

Third Observation. A good deal of attention is often devoted 
to having the machine distinguish correctly between the pattern 
elements that are projected on to various parts of the photo¬ 
field or pattern elements of different dimensions. For example, 
in the case of figure one in the left part of the field, in its upper 
part, the large and small pattern elements of figure one, and so 
forth, must be ‘understood 7 by the machine as one and the same 
pattern element of the ‘figure one 7 . From this paper’s point of 
view, this entire problem has only a technical, and not a funda¬ 
mental, significance. The centring and normalization of the pat¬ 
tern element is a problem that may readily be solved by other 
technical means, and which has no relation to the essence of the 
teaching process. 

Fourth Observation. The compactness hypothesis made it 
possible not only to construct the described algorithms and to 
test them out on the machines, but also to understand the 
fundamental characteristics (peculiarities) of the experiments 
conducted by Rozenblatt, Steinbuch and others, with automatic 
machines specially constructed for this purpose. Thus, for ex¬ 
ample, it is possible to understand why the teaching of pattern 
recognition may be attained in the Perceptron or in Steinbuch’s 
matrix only by adopting the compactness hypothesis, i.e., by an 
a priori assumption that the compact domains are divided in the 
receptor space. 

Fifth Observation. It is natural to inquire: does the com¬ 
pactness hypothesis lie at the basis of at least the simplest pro¬ 
cesses in teaching pattern recognition to the living brain ? This 
question steps outside the limits of the subject of this report and 
of the questions that are of interest to the International Feder¬ 
ation of Automatic Control. For this reason, only brief mention 
is made of the physiological and psychological experiments being 
carried out, within the framework of this study, under the 
direction of Prof. S. N. Braines, at the Academy of Medical 
Sciences of the U.S.S.R. 

Psychological experiments are being conducted with two 
series of blurrings. Each one of these series contains 150 blurs 
specially ‘drawn 7 for these experiments by a machine, in such a 
way that all 150 blurs of the given series comprise a domain in 
the receptor space, and all 150 blurs of the other series comprise 
another domain. 

During the experiments, the children are rapidly shown in 
turn the blurs from these two series, and they are faced with the 
problem of dividing them into two groups. In view of the fact 
that the blurs are deprived of any kind of meaningful significance 
for the children, that their outlines are haphazard and that they 
cannot be separated by any simple, immediately manifest signs, 
the successful conduct of this experiment would be a strong 
argument in favour of the belief that the compactness hypothesis 
in living organisms, too, lies at the basis of recognizing such a 
series of simple ‘senseless 7 patterns. 

Physiological tests, which are closely allied, in their concept, 
with the described psychological ones, are being carried out 
with rats and monkeys. Attempts are being made to develop 
reflexes, in animals, to the appearance of blurs from various 
series, and to judge from the behaviour of the animal as to the 
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degree to which it distinguishes between the blurs which are 
united or divided by only one property: in the receptor space, 
they are correlated with points that form various domains. 

At the moment of writing this report, it is still too early to 
speak of the final results of these experiments, but the pre¬ 
liminary findings are reassuring. 

Teaching Recognition and Automatic Control 

The success of the above-described experiments (see Table 1), 
as well as that of the experiments of Rozenblatt 1 2 , Selfridge 3 ,' 
Stembuch 4 , Bongard 5 and others, makes it possible to foresee 
automatic control systems in the near future. So long as control 
is accomplished by a comparitively small number of regulating 
devices and the control purpose is to maintain the constancy of 
one or of a small number of interrelated magnitudes, the prob¬ 
lems and possibilities of the learning systems are not consider¬ 
able. But, in more complex control problems, where the control 
action is developed by taking into consideration a large number 
of readings (sometimes hundreds or thousands) of control¬ 
measuring devices, attempts to programme control processes in 
advance will inevitably be confronted with considerable diffi¬ 
culties. Control problems, under such conditions, become 
evident only if the possibility of teaching the system recognition 
of situations which arise at the input is kept in mind. 

The difficulties confronted, under such conditions, are similar 
to those that originate in teaching even a qualified engineer the 
execution of the obligations involved in dispatching a complex 
undertaking. It makes no sense to specify, in advance, all the 
situations that might arise in the future, and, to work out in¬ 
structions for the dispatcher’s behaviour under all conditions. 
Experience indicates, however, that a dispatcher trainee, merely 
by observing the work of an experienced dispatcher for a period 
of time, and, under the influence of the simplest system of 
reward (encouragement), can make over this experience, i.e., he 
acquires the ability to come up with correct solutions even in 


new situations which had not been observed by him earlier in 
the teaching process. 

How does this take place? To what degree are the processes 
which occur in this situation similar to those that were attempted 
on the models in our experiments? At the present time, it is 
difficult to answer these questions, and it is not clear as to 
what this study, and studies similar to it, yield towards obtaining 
substantiated replies. However, the value of such experiments 
to the technology of automatic control is evident, inasmuch as 
they make it possible to hope for a substantial expansion in the 
possibilities involved in control systems. Up to the present time, 
in order to carry out various control operations, the systems 
had to be constructed differently beforehand, or else, had to 
have different programmes. At the present time, it is clear that a 
universal control system with a multi-purpose teaching pro¬ 
gramme will be capable of being taught the execution of the 
most varied control problems. The hope has emerged that con¬ 
trol systems in the future, rather than being constructed or pro¬ 
grammed in different ways, will be capable of being taught in 
different ways and ‘educated’ in different ways. It is difficult to 
evaluate the possibilities that this may open for automatic con¬ 
trol technology, if this term is to be understood within any broad 
framework of design. 
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DISCUSSION 


P. Naslin, Laboratoire Central de VArmement, Arcueil (Seine), France 

On page 4 of the paper, the problems of centring and normalizing are 
disregarded, being easily solvable by other means and having no 
rekhon to the essence of the learning process. Does this statement 
hold for the following problems, which are essentially of the same 
nature? 

(1) How can the machine properly orient a character before it has 
identified it? 

(2) If the machine is faced with a small character attached to a 
large blur, how can it eliminate the meaningless blur before it has 
identified the meaningful character ? 

While those problems may be of secondary importance for the 
technical applications of pattern-recognition machines, they seem to be 
relevant to the learning process considered perse, especially as they 
present little difficulty to the human eye and brain. 

M. A. Aizerman, in reply 

The objective of our work was not the development of reading ma¬ 
chines but rather the study of fundamental problems in machine 
imitation of the learning process. We used the recognition of visual 


patterns only as an example of the learning process. Naturally we 
tried to disregard problems connected only with the recognition of 
visual patterns and not with the learning of other kinds of pattern 
recognition, e.g. of recognition of sounds or with development of 
machines for medical and technical diagnostic purposes. As for one 
of the internal problems of reading automata, the centring problem, 
this can be understood either as an independent technical problem or 
can be solved in the course of and in connection with the recognition 
process. However, these problems were outside the limits of my report. 


T. Kasvand, Department of Electronics, University of Southampton, 
Southampton, England 

It is evident that a superior system can ‘understand’ the operation of 
an inferior system. However, it has been said that this superior system 
cannot understand its own method of operation, since that would 
require more information than the system is able to store and handle. 
If there is any basis to this belief there may exist an insurmountable 
gap between the human intellect and the intelligent systems that we 
are trying to devise. Do you feel that this fear is ungrounded, or that 
the speed of computers will compensate for the lack of deeper under¬ 
standing? 
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Is there but one mathematical operation that maps the responses 
of the n receptors to a point in the ^-dimensional space, or is there a 
whole sequence of operations, each in turn dealing with increasingly 
more complex concepts, where the final operation produces the single 
point in the ^-dimensional space ? 

Has any attempt been made to transform the /z-dimensional space 
to a suitably defined 3-dimensional space? This may throw some light 
on the way the brain handles the recognition problem. 

Since there is no well-defined terminology I regret that some of 
my questions may not be clear. 

M. A. Aizerman, in reply 

In his first question Dr. Kasvand touched upon the principal problems 
of possibilities of machines which I intentionally avoided in my report. 
Up to now, to programme for any particular process, a human 
operator had to know how to realize this process and also to under¬ 
stand how it could be realized, that is, he had to be able to compose 
the algorithm—the programme of this process. The idea of the report 
was to go beyond this and to show that the machine can be taught to 
do the same operation that a man can perform even if he himself 
does not understand how he does it (by demonstration). It seems to 
me that this is a way to expand the machine possibilities, and from 
my point of view it is more important to advance in this direction 
than to try to foresee the limits which can restrict progress along this 
line. Any attempt to find such limits requires a more precise statement 
of the problem than we can make at present. 

On the second question I would like to explain that as ^-dimensional 
receptor space we use directly the space of receptor states without any 
intermediate signal transformations, and, in particular, without 
decrease of space dimension in comparison with the number of 
receptors. The concept ‘compactness of pattern 5 is pointless if the 
space is not fixed. The meaning of the compactness hypothesis consists 
in the fixing of this space: namely, we proceed from the assumption 
that the pattern is a collection of points composing a region in the 
space of receptors. Of course, we can choose transformations which 
make this region ‘more compact 5 , e.g. just convex, and in this way 
we can make easier the problem of division of this region. However, 
there are transformations which generally disturb the compactness. 
The problem of distinguishing ‘good transformators 5 requires further 
research or new ideas. 


J. E. Gibson, Control and Information Systems Laboratory , Purdue 

University, Lafayette, Indiana , U.S.A. 

I listened with interest to this preliminary report of the work on 
learning systems which Professor Aizerman’s laboratory is beginning. 
The ‘compactness hypothesis’, which is advanced without proof, 
apparently implies that every pattern constitutes a compact set, that 

is, it can be isolated by hypersurfaces from every other pattern of the 
domain considered. This notion has been previously assumed by 
Rozenblatt, Widrow, Matson and others, but it appears not to have 
been stated explicitly previously. 

The first algorithm of Aizerman for separating patterns by piece- 
wise linear hyperplanes has been employed by Widrow’s Madeline 
among others. The second algorithm, that of the potential hill, appears 
to be new and is an interesting contribution. It appears to have the 
drawback, however, that a digital computer will be required to realize 

it. This adjustment is definitely non-trivial since the shape of the 
potential function will affect the accuracy of recognition and its 
convergence. All of these matters remain to be discussed in the 
future. 

Finally, I feel that the application of learning theory to the 
solution of automatic control problems will be most fruitful in 
an on-line extension of current adaptive control theory rather than 
in the off-line training sequence approach suggested by Professor 
Aizerman. 

We look forward with great interest to future detailed reports of 
the progress of Professor Aizerman and his group in this very difficult 
research area. 


M. A. Aizerman, in reply 

I am very grateful to Professor Gibson for his interest in our work, but 
I cannot agree that our algorithm of random planes concurs with that 
realized by Widrow in his Madeline. The Madeline algorithm slightly 
resembles ours only in appearance. 

The principal feature of our algorithm is that the random planes 
are built as the points are appearing and that the piecewise surface 
is formed by erasing superfluous parts of these frames. As I under¬ 
stand it, Widrow’s algorithm does not contain such a process. 
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On the Theory of Self-tuning Systems with a Search 
of Gradient by the Method of Auxiliary Operator 

I. E. KAZAKOV and L. G. EVLANOV 


Summary 

The paper considers the method of auxiliary operator for the deter¬ 
mination of the gradient of index of the quality of control with respect 
to tuned parameters for continuous self-tuning. A theoretical investi¬ 
gation is given of the dynamics and precision of the extremal tuning 
of parameters of an automatic control system with the utilization of a 
method of auxiliary operator. The effectiveness of the considered 
system of self-tuning with random disturbances is demonstrated. 


Sommaire 

Ce rapport presente la methode de Foperateur auxiliaire pour deter¬ 
miner le gradient de Findice de qualite de commande en ce qui con- 
cerne les parametres accordes pour un accord automatique continu. 
On examine en particulier Faspect theorique de la dynamique ette la 
precision de l’accord extremal des parametres avec cette methode 
dans le cas d une commande automatique. On demontre Fefficacite 
de cette methode dans le cas de perturbations aleatoires. 


Zusammenfassung 

Dieser Beitrag behandelt den Einsatz eines zusatzlichen Operators zur 
Gradientenbestimmung des Indexes der Regelgute fur die angepaBten 
Parameter bei kontinuierlicher Selbstanpassung. Die Dynamik und die 
Genauigkeit der Extremwertanpassung solcher Systeme mit Hilfs- 
operator werden theoretisch untersucht. Die Wirksamkeit eines der- 
artigen Systems mit Selbstanpassung bei Zufallsstorungen wird dar- 


Structure and Equations of a Self-tuning System 

In many cases important in practice, automatic control systems 
may be represented in the form of a generalized system illus¬ 
trated in Figure 1. The controlled plant is characterized by an 
operator of a given structure A(rj), where V is a group of para¬ 
meters for which a priori information is lacking. The system of 
control is described by an operator BQ) which depends on the 
group of parameters f* (; = 1 , 2 ,..., h) which may be adjusted. 


X+Z 


&—{ B$) 


a(h) 


Figure 1 

In actual systems, the entirety of values of each parameter f 
forms a finite multitude H, The input signals of the system an 

disturbances S6 ^ U rand ° m signa1 ’ and Z «> 17(0, randon 
follows - eqUati ° nS ° f the automatic control system are as 


Y = A(t 1 )[V+U] 

V=B($)e (1) 

e=X+Z-Y 

In order to assure high quality operation of the automatic 
control system it is necessary to achieve tuning of parameters 
of the operator B(g) in the presence of variation of the char¬ 
acteristics of the input useful signal X(t) 9 of the characteristics 
of disturbances Z(7), U(t ), and also in the presence of variation 
of parameters rj of the operator of the controlled plant. 

In order to construct a circuit for self-tuning, an index of 
quality 7 of the automatic control system is introduced. The 
index of quality 7 is a function, or in the general case it is a 
functional of tuned parameters. Ordinarily the index of quality 
7 is computed on the basis of error s of the system: 

I = Nf(e, c) (2) 

where N is an operator or a functional, f(ej) is a function of 
the error of the system depending upon the error e and the 
tuned parameters f. 

In order to adjust the parameters of the system use is made of 
the broad possibilities offered by the method of steepest de¬ 
scent or gradient, a discussion of which is considered by Feld- 
baum 1 . Applying this method for tuning parameters £ one 
has: t 

£ = /lgrad7 ( 3 ) 

where X is the scalar multiplier, and f is a vector function of the 
velocities of tuned parameters. In accordance with the gradient 
method the self-tuning system assures the tuning of parameters 
S for the °P tlmal value of index of quality 7 0 . In the general case 

7 0 =inf7(0 or J 0 = s u p7(f) (4) 

%i eE i fieS, 

In the particular case when the lower (upper) boundary of the 
multitude & is attained within E i9 


For a complete description of the circuit for self-tuning it 
necessary to determine the method of computation of the con 
ponents of the gradient from the quality index for the tunc 
parameters. In the given investigation a method is applied whicl 
m the following is termed the method of an auxiliary operate 
Its essence consists of the following. 

If the information on operators BQ) and AM is know 
a priori it is possible to construct a certain auxiliary operate 
C (?, ij) whose application to the error of the tracking systei 
makes it possible to compute the components of the gradier 
vector. ° 
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The derivative b I/b is computed by the direct differenti¬ 
ation of the expression (2) assuming that the operators N and 
differentiations with respect to are commutative. 


8/ a/(8,c) ,as 8/(e,o 

0c ; Se s£; JV 0ft 


( 6 ) 


The derivative bs/b will be calculated by differentiating the 
system of eqns (1). The derivative of the error e with respect 
to is equal to 


de_ SY 

a«r 8 fi 


(7) 


is one of absence of a priori information on parameters rj. In¬ 
formation on parameters of the plant may be obtained on the 
basis of application of a tracking system, certain aspects of 
whose application were considered by Margolis and Leondes 2 ’ 3 . 

The structure of the operator of model A (f) is based on the 
utilization of a priori information on the plant. The entirety 
of parameters f of the operator of the model is tuned for the 
value rj. The circuit of the tracking model is constructed quite 
analogously to the circuit for tuning. Introducing an index 
of approximation J of parameters f into parameters rj , 

J = L<j> (A) (13) 


since the input signals X(t), Z(t ) do not depend upon ft. The 
derivatives of the output signal are computed: 

( 8 ) 

Excluding from (7) and (8) bY/b and transforming, one 
obtains: 

||=-[i+^oof?(orM( n )^p (9) 

Introducing the designation 

C^ll+AWBOOy'Air,)?^- ( 10 ) 

one writes: 

g£=-C, (1J,0« (11) 

or 

grad e~-C(fi,£)e (12) 

where C (r/,f) is an auxiliary operator-vector which is completely 
determined by the operators A(r]), Thus, the gradient of 
the quality index for tuned parameters is determined by eqns (6) 
and (11). 

The method of auxiliary operator requires an a priori 
knowledge of information on the system, and this somewhat 
restricts its generality. However, there exists in technology an 
area of applicability of the method inasmuch as the predominant 
majority of created automatic control systems can be described 
mathematically. 

The advantages of the method are the absence of trial load 
changes and the possibility of accelerating and simplifying the 
process of computation of the gradient components. In self¬ 
tuning systems with a search of gradient by the method of trial 
load changes, a priori information on the plant, other than the 
knowledge of the band pass of the system, is not required. This 
permits a correct selection of the frequency of the trial load 
changes and constitutes the advantage of this method. However, 
its basic shortcoming is the limited quick response imposed by 
the finite band pass width of the system. In the considered 
method the band pass of the mathematical model of the system 
(operator C) may be artificially broadened by changing the 
time scale of the solution. The possibility of simplifying the 
process of computation is based on the substitution for a com¬ 
plex operator C of an approximate and simpler expression. 

The auxiliary operator C (rj 9 g) depends upon the para¬ 
meters of the plant and the system of control. A typical case 


where L is an operator for computing the index /, and (j) (%) is 
a function of the error. The error is determined by the relation¬ 
ship 

s 1 = Y M (t)-Y(t) (14) 


Here Ym(J) is an output signal of the model determined by the 
expression 

Y M (t) = A(QV (15) 

The change of the parameters of the model is carried out by 
the method of steepest descent: 

£ = A*, grad J (16) 

where is a scalar multiplier, and £ is a vector function of the 
velocities of the tuned parameters of the model. 

In order to determine the components of the gradient one 
applies the method of auxiliary operator: 


dj_ a* («,)&, 

wr L ~^r e?r (17) 

Differentiating the relationship (14) with respect to £$, one has: 


de t 


05m 

SG 


0 

0ft 


A(0V= 


mo 

0ft 


V 


(18) 


hence it follows that the auxiliary operator in a given case is an 
operator-vector (7(£) with components 


Thus 


G,( 0 = 


mo 

eft 


grad J = L G(Q fJ 


(19) 

( 20 ) 


Equations (13), (14), (15), (16) and (20) describe the operation 
of the tracking model. A useful output of the circuit of the model 
is the entirety of parameters of model £. For ideal operation 
of the model £ = rj. An actual model assures the attainment of 
parameters £ close to values rj , and therefore, strictly speaking, 
in the operator C it is necessary to replace parameters rj by £. 

The complete block diagram of the self-tuning system in 
accordance with eqns (1), (3), (6), (11), (14), (15), (16) and (20) 
is presented in Figure 2. The schematic diagram was proposed 
by Evlanov. 

The structure of the self-tuning system contains three cir¬ 
cuits : the basic circuit of the system, the circuit of the tracking 
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Figure 2 


model, and the circuit for tuning the parameters. The circuit of 
the tracking model assures the reception of information on the 
parameters of the operator of the plant. In the following the 
operation of the circuit of the tracking model is assumed to be 
ideal, that is, f = rj. The circuit for tuning the parameters as¬ 
sures the tuning of parameters of the control system in accord¬ 
ance with the given optimal value of the quality index of the 
system. 

Investigation of a Self-tuning System in a Quasi-stationary Regime 


where m e is the mathematical expectation, and s° is the centring 
component of magnitude e. In the function of the error f(sj) 
we shall also factor out the mathematical expectation 

fM=Mf(e,Z)+f°(e,Z) ( 22 ) 

where M is the operation of mathematical expectation, /° (ej) 
is the random centred component. 

The quality index of control I introduced previously may 
now be presented as: 


A typical regime of operation of a self-tuning system is the 
case of a change of parameters r] of the operator A(rj) of the 
plant and of the characteristics of external random disturbances 
X, Z, U which are slow compared with the duration of tran¬ 
sient processes in the basic circuit of the system. In this case 
it is permissible to consider the circuits for tuning parameters 
and the tracking model on the one hand, and the basic circuit on 
the other hand, as being autonomous, since the tuned para¬ 
meters | and parameters rj may be considered as constant during 
the time of process control in the basic circuit. It is also assumed 
that the tracking model carries out its functions in an ideal 
maimer. Under these conditions the process of self-tuning of 
parameters f of operator B(£) is investigated in the vicinity of 
extremum of the quality index I. 

The presence of extremum in the quality index I of the 
system with respect to all or several of the tuned parameters is 
an important property of self-tuning systems which permits 
them to be tuned for an optimal regime. If the error of the 
system s or another characteristics does not possess extremal 
properties, then it is possible to construct an extremal quality 
index by artificial means depending upon the direction of the 
target of the automaton. This will be shown below by an example 
of a typical tracking system. For the time being, however, it is 
assumed that the quality index I possesses extremal properties. 

The random error s of the basic circuit can be expressed in 
the form 

s = m e + e° ( 21 ) 


I* = NI* + Nf° (fi£) (23) 

where the designation I* is introduced for the statistical 
quality index of control 

I* = M/(e,{) (24) 

Computing the components of the gradient of the quality index 
of control by parameters one obtains: 


dI -=N^ + N d ^ +N ^ + N^ 


0£i 


0m £ '"W 06^06 


(25) 


Representing the statistical quality index I* of control in the 
vicinity of the investigated extremum by a quadratic form in 
terms of deviations u t — — f i0 of parameters f,- from the 

optimal values f 0 , and considering that 


07* 

3f«. 


=0 


ft=fto 


values of bl*/b the expressions: 


S7^_ " 2. 

Ki h 2 


,2 r* 


0 2 i 

L 


U: 


(26) 


Differentiating expressions (24) twice with respect to para¬ 
meters fj- and utilizing a system of equations of the basic 
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circuit of control for optimal parameters of operator i?(f), 
one computes the coefficients 

- g 2j * - 

AWJo 

in the form: 


~ 1 . f ft7(« 0 .e 0 ) 

MMj] 1 88^ 


(Cy 0 £o) (Qo s o) 


9/(«o.go) 

0e o 


(C jO C; 0 £ o) + 


8 2 f (e 0 , So) 
08 QSj 


(C i0 e 0 ) 


8 2 /( £ o^o) 

6£8 i, 


(CjO £ o) + 


9 2 /(e 0 , g 0 ) | 

8«.0 h J 


(27) 


where C i0 (| 0 , rj) are the auxiliary operators (10) for optimal 
values of parameters f i0 . 

Introduce the designations: 


1 

2 


0 2 i* 

AW Jo' 


Taking into account also that 


(28) 


8 m, 

8fi 


= -Cjm,, 


08 ° 

8£> 


= — C:S 0 


(29) 


the formula (25) is written for the components of the gradient of 
the magnitude I in the form: 


ar « fifO a/0 grO 

e5=M, <L. c ‘ m -- N -k c “ +N lk <30) 


Substituting the expression (30) into formula (3), one obtains 
a system of equations of the circuits for tuning the parameters 
in a scalar form: 


« ft r 0 ft /° ft r 0 

^ i =2iVS i « y » J --^0^C i m-2iV-^C i 8 o +AIV-U 

‘(31) 

From this one obtains a system of linear equations for the de¬ 
termination of mathematical expectations of deviations m Ui of 
tuned parameters from the optimal values: 


m u 


-AN E a i} Uj= 


i0 


(32) 


In order to determine random components of deviations of 
tuned parameters u? one obtains the following system of linear 
equations: 


u?-XN E M °a 0 -= -XN 

j =i 


' 8 /°' 

6m. 


C io m eo 


-XN 


8 /°" 

0e° 


C^+XN^ 

0 u Si 


(33) 


An analysis of approximate linear equations (32) makes it 
possible to evaluate the stability of the process and to determine 
the systematic errors of self-tuning of parameters £ z *. In partic¬ 
ular, if the basic circuit of control is stationary and possesses 
astatism of the kth order, then for stationary random disturb¬ 


ances Z and U, and for an additive component of the useful 
signal X in the form of a polynomial of the kth. order, the left- 
hand parts of eqns (32) are stationary. In this widely encountered 
case the stability of self-tuning of the parameters is characterized 
by the properties of the characteristic equation. In this case the 
investigation of stability is carried out by ordinary means. In the 
general case the systematic components of the errors of para¬ 
meters are computed by equations: 


n 


m Ui (t)=~ E 

j= i 


gij(U 0 Sjo (0 

0 


(34) 


where g i0 (t, t) are the weighting functions of the system of eqns 
(32). If £ i0 = const., then the systematic values of errors of tuning 
of parameters m Ui — 0. Dispersions of the errors of parameters 
are determined on the basis of the system of eqns (33) by 
applying the theory of transformation of random functions 4 . 

From the analysis of stability, duration of transient pro¬ 
cesses of tuning, and evaluation of the precision, one chooses 
the coefficient X and also other characteristics of the tuning 
circuits. 

The final evaluation of mathematical expectation of the 
error in the basic circuit of the system under the action of 
self-tuning circuits is obtained by the formula: 

n 

m c = m e0 + E m tt (35) 

i = 1 

where m 8o is the mathematical expectation of the control error 
of control € for an optimal value of parameters. 

The magnitudes m 8 . are determined by the expressions: 

~ Cio (£o) 


where C io (£ 0 ) are the auxiliary operators for optimal values of 
parameters £ l0 and the magnitudes bi are equal to 


bi = 


a m 
. . 


m P , 


(36) 


The evaluation of dispersion of the error in the basic circuit 
is computed by the formula: 

D £ = D eo + 2 E K 0 «+ E K>J ( 37 ) 

i= 1 i,j= 1 


where D 8q is the dispersion for optimal values of parameters f io , 
K PnPli K PsPi are the coefficients of correlation of random com- 
ponents of the control error £*°, and the magnitudes are 

equal to e? = - uf [C io (£ 0 ) m £ J (38) 


Linear Tracking System with One Tuned Parameter 


The application of the method to a linear tracking system, 
with one tuned parameter, is now described. In tracking systems, 
as a rule, the index of control quality is assumed to be the 
second initial moment of error e. This magnitude does not 
possess extremal properties with respect to parameters f corre¬ 
sponding to the change of input random actions X, Z, U. 

Now consider an example of a tracking system having the 
following characteristics: A(rj) = J5(£) = g l9 X = at, U — 0, 


m 2 = 0, s z = 


Dm P 

n (co 2 +/? 2 ) 
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and values of parameters given by rj t = 10, <2 = 0 , 1 , D z = 10~ 4 , 
jj — 100. The second initial moment of error a in a stabilized 
regime is equal to: 



This relationship has no extremum with respect to parameter 
In the theory of optimal filtration the magnitude £* = 
e — Z = X - Y is considered as an error. The second initial 
moment of this magnitude possesses extremal properties. Thus, 
under the conditions of the preceding example the magnitude 
cc* is equal to: 


to a low frequency filter. Now consider the quasi-stationary 
regime of self-tuning of parameters. Eqn (31) of tuning of 
parameter ^ stated with respect to deviation u x assumes the 
form: 


[(TD +1) D - AaJ u t = - 2 2m £0 [C 10 (0) + 0 (£>)] eg 


where 


— D^ 0 + 21vm,Z° 


(42) 


«1 =M{lC 10 (D) B° 0 f + [e 0 (Cj o (D) eg)]} > 0 (43) 


o^oflO 5 ! ocoj-10 5 


a _, DzV jji 

Z\n\ €i*ii+fi 


(40) 


This function has an extremum with respect to parameter 

It is possible to measure directly the magnitude £* in tracking 
systems using a priori information on the statistical properties 
of the input useful signal and the disturbances. In practice it is 
possible to measure the error e and the signal Z 1 = Z x (Z,X) 
related to Z. For instance, the function Z x may be obtained by 
filtering with special filters the input signal X + Z and utilizing 
the information as the spectrum of the frequencies of the 
disturbance Z, as a rule, is substantially broader than the spec¬ 
trum of the useful signal X. Then the function Z 1 will possess 
characteristics which are close to the characteristics of the 
function Z. 

Measuring the magnitudes s and Z x it is possible to formulate 
artificially a quality index having an extremal characteristic with 
respect to the gain of the correcting circuit £(£). For this the 
function of the error is assumed to have the form: 


f(s,0=s 2 + il/(^ 1 )Z 2 1 (41) 


The function may be chosen in a specific case, for instance, 
from the condition of proximity of the extrema of functions 
M [e—Z] 2 and M [e 2 + yj(£j) Z 2 ] with respect to parameter 
for a statistically prescribed input signal. 

As an illustration of the method of prescribing a function 
let us consider the case of good filtering when it is possible 
to neglect the component X in function Z x . Let us determine 
V>(£i) = v£ lt where v is a constant coefficient computed from 
the condition of proximity of the values of parameters £ 10 for 
extremal values of the functions a = Ms 2 + v£, D and a* — 
M(s— Z) 2 . ~ 

In Figure 3 there are presented graphs of functions a and ocl 
corresponding to the minimal value and computed for the 
preceding example. For v = 0-1 the minima of the functions 
(curves with an index 1 ) coincide closely, and the optimal value 
of parameter £ 10 = 3-0. The change in a sufficiently broad range 
of probability characteristics of disturbance Z, useful signal X 
and parameter rj leads to a distortion of the form of the curves 

? and ? E - However ’ ^ minima coincide, but are not reached 
or other values of parameter f 10 as shown in Figure 3. In 
figure 3 the mdex 2 denotes curves for D 2 = 10~ 3 and the pre¬ 
vious values of other parameters. 

In Figure 4 there is shown a schematic diagram of a linear 
tracking system with tuning of the gain & for «>(&) = 

The function Z } is separated with the aid of a band pass filter 
or a filter of high frequencies. Then the signal is supplied to a 
square wave generator and a circuit with gain v£ lt and then 


Figure 3 




brom these one obtains the following equation for the de¬ 
termination of mathematical expectation m ul : 

{TD 2 4-D — Xay) m Ul = —Df 10 ( 44 ) 

For X < 0 the stable process of tuning is assured. When one 
equations^ 6 C6ntred rand ° m COmponent one obtains the 

[TD 2 +D-X ai ] u°= -2 Xm S0 [C lo (0) + C 10 (D)] e° 0 

+ 2Xvm Z\ f451 
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The magnitude m Zl may be set equal to zero by proper selection 
of the corresponding filter. Taking this into account and also 
utilizing expressions for e 0 ° in terms of X° + Z°, one obtains 
from eqn (45) 

u° 1 =<S> 1 (D)(X 0 +Z°) (46) 

WhCre ® rm -2Am go [C 10 (0) + C 10 (P)] 

1 () (TD 2 + D — Aa 1 )[l+A (D) B 0 (£>)] 


(47) 


In this case, for computing the dispersion of parameter u x in a 
stabilized regime, one obtains: 


D 


Ml 


m 00 

|<D 1 (ico)| 2 [S x (o J )+S z (co)]d® 

— 00 


(48) 


where S x and S z are the spectral densities of random functions 
X and Z. For £io = const, the magnitude m Fo — 0 in the sta¬ 
bilized regime. In this case the systematic error of a tracking 
system with self-tuning in a stabilized regime of operation is 
equal to m e = m Eo , that is, equal to the systematic error for an 
optimal value of parameter £ 10 . The random component of the 
error tracking is equal to: 


£° == 


1-f 


MCA 


1 + A(D)B 0 (D) 


(A 


1 


1+A (D)B 0 (D) 


(X° + Z°) (49) 


where the magnitude b L according to formula (36) is given by 


dB 0 (( 0 ) 
1 9£io 


m 8 0 


(50) 


good effectiveness of tuning. Thus, the mathematical expectation 
of tuned parameter is equal to = ft 0 , and the dispersion 
of the error of tuning computed by formula (48) is given by 
A, = A, = 4 x 10“ 7 . From these calculations it follows that 
the maximum relative error of tuning the parameter f l9 is equal 
to 6*3 x 10~ 2 per cent. As regards the tracking error by the 
follow-up system, the mathematical expectation of this error in 
tuning coincides with the value of this magnitude in an optimal 
system m s = m €o = 0*33 x 10~ 2 . 

The dispersion of the tracking error in a self-tuning system 
computed by formula (51) coincides with a precision to three 
significant figures with a value of dispersion of the error of 
tracking in the optimal system D & & D So = 2*31 x 10~ 5 . Thus, 
in the considered example the self-tuning system with the utiliza¬ 
tion of the method of auxiliary operator assures an effective 
tuning for the minimum of the second initial moment of error 
in the presence of random disturbances. 

Conclusion 

The considered scheme of a self-tuning system may be 
effectively utilized both for the direct control of plants and the 
synthesis of automatic control systems during their design. The 
advantages of the system of self-tuning utilizing the method of 
auxiliary operator are: relative simplicity of achieving tuning 
circuits, effectiveness of operation in the presence of disturb¬ 
ances, and the possibility of obtaining high values of quick 
response. 
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In computing the dispersion of error e one obtains the formula: 

biA(ico ) 1 1 _ 2 

+ I +A(icD)B(ioo) x ^ lC0 'j l + A(icD)B(ico) 

[S x (a)) + S z (a)y]dcD (51) 

The calculations carried out for a tracking system {Figure 4) 
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DISCUSSION 


P. Eykhoff, Technological University , Delft , Netherlands 

Professor Kazakov and Dr. Evlanov presented an interesting contribu¬ 
tion to the theory of self-adjusting systems. Basically their approach 
is an application of Meissinger’s parameter-influence coefficients 1 ' 2 
to a type of problem for which, to the best of my knowledge, they 
have not yet been applied. The auxiliary operators introduced in the 
paper can be considered as instrumentations of Meissinger’s sensitivity 
equations. 

Besides the general questions related to speed of convergence and 
stability, there are to my mind a few questions that have to do with 
assumptions made in the paper. 

First it is ‘assumed that the tracking model carries out its function 
in an ideal manner’. Two other authors, Margolis and Leondes, 
employ the same type of approach to parameter tracking. From their 
work one notices that under favourable conditions and for the simple 
case of one parameter to be adjusted it still takes an interval of the 
order of five process-time constants to bring the model parameter 
close to its desired value. Under certain conditions even stability in 


the small is not assured. For this reason I am a bit worried by this 
assumption of ideal tracking. 

In the second place I am concerned about the validity of the 
assumption stated as ‘representing the statistical quality index I* of 
control in the vicinity of the investigated extremum by a quadratic 
form in terms of deviations m = ft —ft 0 ...’. No one will question 
that, mathematically speaking, an ordinary extremum may be ap¬ 
proximated by a quadratic form. For the simple parameter tracking 
problem of Margolis and Leondes, however, one finds the relation 
of Figure A between I* (or better J*) and (ft — ft 0 )/fto- This leads 
to the question whether in an engineering sense the region over which 
the approximation holds may not become trivially small. 

Finally, I would like to pay some attention to the auxiliary opera¬ 
tors. 

For the sake of simplicity let us take: 

°‘~~W 
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An arbitrary example with the parameter & in the denominator of the 
transfer function will show that the operators C t may become rather 
complicated. This is not quite the case if the auxiliary operator is 
constructed according to Meissinger 1 ’ 2 . In that case the inputs of the 
auxiliary operators are the dependent variables from the model A (|), 
i.e. the model A (£) is a part of the auxiliary operator. A further 
simplification is obtained when instead of A (£) a ‘generalized model’ 
according to Figure B is allowed 3 . Then there is no need for separate 
auxiliary models and one arrives at the interesting model-adjustment 


determination of human and other system parameters. Proc. Joint 
Computer ConfLos Angeles (May 1961) 645-660 
6 Gra Y pe > k * K. The analogue solution of some fundamental 
analysis problems. Communication and Electronics; Trans Amer 
Inst. Elect. Engrs. Pt. I, 52 (1961) 793-799 

J. Zaborsky and D. Gorman, Washington University, St. Louis 30 
Missouri , U.S.A. ’ 



u(t) 


Figure B 

schemes of Clymer 4 . ■ and Graupe 6 . Under some conditions even a 
fast, monotone convergence of the error e to zero can be assured 6 
irrespective of the type of process-input signal. Additive noise 
however, leads to a bias in the adjusted parameters. Means to eliminate 
this drawback will be reported in the near future. 
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(1) The paper consistently uses the terms ‘operator’ and ‘functional’ 
without stating the limitations imposed on these operators by the 
manipulations used in the paper. The most striking example is that 
N and L are simply identified as ‘an operator or a functional’, which 
enter into the indices of control and appear to be restricted linear 
instantaneous operators. For a deterministic system N and L may 
seem limited to scalar constants; for a random system N and L may 
be expectation operators. Time integrals such as integral square 
criteria are not apparently permitted, however, since these change 
the derivatives of / and J into functional derivatives, thus altering the 
rf/t\ re ° f the problem - the ot her operators, especially A (rj) and 
B (0, seem to be limited to be linear and dependent only on instan¬ 
taneous values of the arguments. 

(2) What is the validity of the stability analysis in the paper 
considering that A ( V ) was substituted for A (f). This could introduce 
delays and inaccuracies which may strongly affect stability. 

(3) With regard to the example, what would make it necessary to 
use the second initial moment as a performance criterion? 

(4) Once the assumption is made in the example that noise Z can 
be essentially separated from useful signal X it would appear logical 
to use this filtered X signal directly for control and avoid the complex 
secondary structure involving Z x , as introduced in the paper and 
shown in Figure 4. This difficulty does not seem limited to this specific 
example but appears general for the method since only signal s would 
be normally available for direct measurement with the assumptions 


J. D. Roberts, University of Cambridge, Cambridge, England 

The method of the auxiliary operator will, I believe, have important 
applications in designing self-adjusting control systems. A similar 
idea is used in the sensitivity function used by Olsum in model reference 
adaptive systems. I also mention my paper, published in the I.E.E. 
m 1962, which embraces this kind of system. My first point is that the 
auxiliary operator is realized by a network (called the ‘auxiliary 
network’) which is related to the original network in a precise way. 
Consider the example given by the authors: 

An auxiliary network is easily constructed in which signals cor- 
t0 E and Vm the origina -l network have values ds/6^ and 
The l0 ° P iS duplicated and a connecting link is provided 
Is i 1 realized by a straight connection. Non-linear characteris¬ 
tics can also be treated. In particular, if a saturation characteristic 
is introduced after the signal V, a switch must be introduced after the 
signal o Vlo£ v This must be in the break position at times when the 
non-linear element is saturating. As a consequence of a discussion 
with Professor Pugachev, I point out that the signal de/d^ can also be 
obtained by a connecting link which adds the signal Vto the input of 
the auxiliary B (£) (the dotted connection) instead of adding the 
signal s to the output of the auxiliary B (£). However, this alternative 
fails when non-linearities are introduced. 

Finally, a point concerning the required accuracy of the auxiliary 
A (rj). In this respect, the method is especially powerful with cascade 
or open-loop controllers, because the accuracy is not critical. As an 
extreme case, consider the open-loop control of a system with known 
dynamics A, but unknown gain rj lm The auxiliary network is shown 
with an inaccurate gain rjf*. The criterion of zero mean product of the 
signals is unaffected by an error in rjf*. More specifically, the value 
of satisfying this criterion will be intermediate between the time 
optimum and the optimum for the inaccurate %*. 
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P. Kokotovic, Institute for Automation and Telecommunication s Bel¬ 
grade, P.O.B. 906 , Yugoslavia 

Taken generally, the two basic problems in the synthesis of self- 
adapting systems, namely process identification and parameter ad¬ 
justment, can be solved using gradients of the properly chosen index 
of quality / and index of approximation J. Applicability of such an 
approach depends to a great extent on the effectiveness of the method 
by which the components of the vectors grad I and grad J are deter¬ 
mined during the operation of the system. In this regard the parameter 
influence technique offers promising possibilities 1-10 . 

The paper by Kazakov and Evlanov represents an excellent theo¬ 
retical generalization of the application of the parameter influence 
technique to the synthesis of self-adapting systems 11 * 12 . It should 
be pointed out, however, that the structure shown in Figure 2 of the 
paper increases in complexity as the number of the parameters to be 
adjusted increases. We would, therefore, suggest a procedure 8 * 9 which 
avoids this difficulty, at least as far as linear systems are concerned, 
and enables all parameter influences to be computed simultaneously 
by means of one single structure that could be called the parameter 
influence analyser. 


The Structure of the Parameter Influence Analyser 


Let the system transfer function W = W (s, q) depend on para¬ 
meters qu l — 1,..2 ni. Let Wi --- Wi (s, q \) be the transfer function 
of the it h system clement containing the parameter qi. Then the system 


equation 


Y(s,q) = X(s)W(s,q ) 


( 1 ) 


can be differentiated as follows 


8y__ dlnW 3In W t 
0 In q { d In W t 01n q t 


z a mr z (3) 

are the influences of the relative changes of parameters on the system 
response y (r, q). According to eqns (2) and (3) the influence m (t ) is 
computed from the response Y (, s , q) and the transfer functions 


In W In W{ 

P,=r~Fi 7 and F i=- i- 1 

1 In HP; In qn 


It can be shown that the transfer function Pi determines the position 
of the point Si in the structure of the system W (s, q). We shall call 
these the sensitivity points. It follows that all parameter influences 
m (t ), i — 1, ...,2 m, can be obtained simultaneously by means of a 
single structure that consists of one main part identical with the system 
structure and several additional parts whose transfer functions are F{, 
and which are connected to the main part at the corresponding 
sensitivity points Si. 

To illustrate this procedure we use the system shown in Figure A. 
Positions of sensitivity points in this general multi-loop feedback 
structure are indicated in Figure B. The parameter influences are 
obtained at the outputs of Ft. 

It should be emphasized that F% are usually of a simple structure. 
In the most important practical case, i.e. when the parameter qi is the 
gain factor of the element W{, F% becomes equal to unity. Hence, the 
parameter influences are obtained directly at the sensitivity points and 
the structure of the parameter influence analyser becomes identical 
with the structure of the system. 

It follows, therefore, that in many practical cases blocks C v ..., C n 
in Figure 4 of the paper can be substituted by a single structure which 
usually is no more complex than the structure of the system itself. 
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The similar case applies to blocks G 1 ,...,G n .ln this way the realization 
W “ h adJ ““ bk " * 

■Application to Automatic Design 

Such a procedure has been applied also to the automatic design of 
control systems using a general purpose analogue computer 14 . Basic 
diagram of the analogue computer circuit is shown in Figure C 

Operating modes of the blocks 1.5 are automatically switched in 

the sequence operate-hold-init. condition’, while the operating 
sequence of blocks 6 and 7 is ‘hold-operate-hold’. The computing 
circuit can be still further simplified if a repetitive computer equipped 



t(sec) 


with servomultipliers is used. Figure D is an example of op timiza tion 
process for a system having a structure as in Figure A. Parameter 
values are rapidly found by the computer, even in cases where the 
starting point falls in an unstable region. 

Our experience in solving similar problems confirms the opinion 
of the authors that the method described can be successfully used for 
automatic design of control systems. 
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Problems of Continuous Systems Theory of 
Extremal Control of Industrial Processes 

A. A. KRASOVSKI 


Summary 

An extremal control of a continuous industrial process is discussed. 
A control system maintains the extremum of some index of the 
product quality. Equations of continuous extremum control systems 
are analysed. The effects of delay and disturbances on the control 
processes are determined. 

Effective circuits for the production parameter control are dis¬ 
cussed. Possibility and expediency of introducing parametric self¬ 
adjustment of extremum control systems for complex industrial pro¬ 
cesses are shown. 

Sommaire 

On examine le probleme de commande automatique pour obtenir l’ex- 
tremum d’un indice de qualite d’un processus industriel continu. On 
analyse les equations generates d’extremalisation les effets des re¬ 
tards et des perturbations. On examine les possibilitds et l’efficacite 
de certains sch6mas de commande automatique extremalisante pour 
des processus industriels complexes. 

Zusammenfassung 

Die Extremwertregelung von Maschinen, Geraten und Reglerein- 
stellungen, die gemeinsam einen kontinuierlichen industriellen Prozefl 
bilden, wird besprochen. Eine Regelung sorgt fur die Einhaltung des 
Extrems eines Indexes der Produktqualitat. Es werden Gleichungen 
fur kontinuierliche Extremwertregelungen untersucht. Die Auswirkun- 


gen von Verzogerungen und Storungen auf die Regelung werden er- 
mittelt. 

Brauchbare Regelkreise fur die Fertigungskennwerte werden be¬ 
sprochen. Die Moglichkeit und die Brauchbarkeit der Selbsteinstellung 
der Parameter bei Extremwertregelungen wird fur komplexe Industrie- 
prozesse gezeigt. 


Many continuous industrial processes lend themselves to the 
following plan. There is available some quantity n of adjustments 
or controls of machines, apparatus, regulators securing an 
industrial process. The flow of the industrial process and the 
output parameters depend on the coordinates of the adjusting 
or control elements (adjustment parameters). 

Together with the control adjusting element coordinates the 
output parameters are affected by various disturbance factors 
(change of material parameters, wear of machines and tools, 
temperature and moisture variations and other factors). 

The output parameters are controlled continuously or dis¬ 
cretely (but with sufficiently small intervals of discontinuity) by 
special measuring devices—output parameter information 
transmitters ( Figure 1). Influenced by disturbance factors and 
also by random variations of adjustment parameters the output 
parameters are subjected to continuous variations. 



Disturbance 

effects 


Figure 1. Layout of an extremal control system of an industrial process with a single criterion of production quality 
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Even though a practically ideal adjustment of the producting 
system, ensuring the industrial process, is initially attained, after 
some tune the disturbance factors will bring forth considerable 
changes in the output parameters. In order to prevent the drop¬ 
ping out of the output parameters from the established tolerances 
(scrap output), adjustment and tuning of the production system 
is necessary. Various means of automation of these oper¬ 
ations are possible. If it is precisely known which parameter 
and to what extent is affected by one or another controlling 
element, the usual feedback principle may be used (control 
by deviation). For this it is necessary first to smooth the 
results of measurements in order to eliminate overshoots 
o t e system in the presence of small, random deviations within 
the tolerance limits. Methods of such automatic processing of 
information may be set up, based on the widely utilized methods 
of non-automated statistical control 2 . The measured and 
smoothed signals of output parameter deviations are conveyed 
to the performing arrangements and cause changes in the 
controlling element coordinates. Such systemsare sometimes 
called statistical automata 3 . 

Undoubtedly the introduction of statistical automata will 
prove to be an important step in the automation of industry 
However, a necessary condition of their application must be a 
sufficiently complete a priori information on the character¬ 
istics of the industrial process. In many cases this information 
is absent, and even if it is available during the initial period of the 
systems adjustment it loses authenticity in time, due to the 
change in properties of the industrial process. 

Under these conditions the application of usual, non-self- 
cor firol i°°ps (statistical automata) becomes impos- 
sible. In these cases it is expedient to utilize an extremal control. 

The present work is devoted to the investigation of some 
possible circuits of extremal control systems with continuous 
industrial processes and some questions of the theory of these 
systems. It is a development of earlier work by the author 1 . 

For the realization of an extremal control a quality 
output (production) index Q is selected, having extrema at 
wanted values of product parameters. Such an index may be, 
or example, the sum of the squares of deviations of the output 
parameters from the standard values. The quality index Q 
is determined by a computer in diagram Figure 1) based on 
information transmitter data on current values of output 
parameters. To secure the basic function of the system-main¬ 
tenance of the quality index at the extremum level, search 
oscillations are necessary. Natural high frequency random 
oscillations, as well as artificially produced oscillations of con¬ 
trolling elements, may be employed as search oscillations. 
Naturally the first method is preferable, since it is not linked 
with any increase of high frequency fluctuations of the pro¬ 
duction parameters. 

In order to make use of natural oscillations as search oscilla¬ 
tions, it is necessary to measure them. The measurement of 
search oscillations is done by information transmitters for these 
oscillations (.Figure 1), which measure controlling element 
oscillations and disturbance effects transmitted to them. 

The measured search oscillations are transmitted to a 
simulator or a dynamic model of the industrial process. The 
purpose of the simulator is to transform the search oscillations 
in the same manner as these oscillations are transformed in a 
real process. For many industrial processes the simulator may 
be carried out in the shape of a delay line. 


The output signals of the simulator are transmitted to the 
multiplying elements, to the other entrances of which is trans¬ 
mitted the computer signal which is proportional to the current 
value of the production quality index. The output values of the 
multiplying element are smoothed by the low frequency filters 
and are transmitted to the entrances of the control devices 
which move the controlling element. 

If the quality index deviates from the extremum value, then a 
component correlated with the search fluctuations appears at the 
computer output. Values of the mathematical expectation of the 
duplicating links signals differing from zero then appear. Slowly 
changing signals are separated out by the low frequency 
filters and start the control devices. The controlling elements 
act on the production parameters in the direction approaching 
the extremum of the quality index. 

The values of control parameters, together with the disturb¬ 
ance effects transmitted to them, are designated as J v (» = 1, 

2 >•••>”)• Ea ch control parameter brought forth has three 
components. 

X V =X* + 5X V + 5X VW 

Here X v *, working elements, are output values of the ex¬ 
tremal control part of the system; dX v are search elements 
for which it is expedient to utilize high frequency controlled 
effects transmitted to the control parameters, and 6X VW are 
uncontrolled disturbance effects transmitted to the control 
parameters. 

The current value of the production quality index in 
general is a function of indicated control parameters and 
disturbance effects f lt f 2 , according to transmitted con¬ 
trol parameters. 

When the transient process characteristics are described with 
sufficient accuracy by time delays, then the current value of 
the production index is expressed by the function of preceding 
values of indicated control parameters and disturbances effects. 

e = e[JSf 1 (t-T 1 ),...,Z n (f-T„), f t ./ m ] (l) 

The selection of the composition of control parameters 
must conform to the following condition. To each set of 
permanent control parameter values must correspond a definite 
(with an accuracy up to the level of noises) set of production 
parameter values. In other words, in a static regime and with 
absence of noises a simple conversion of control para¬ 
meters into production parameters must be realized. It should 
be noted that no mutual unilateral conversion is required, so 
that the number of control parameters may greatly exceed the 
number of controlled production parameters. 

In virtue of one-sided-unilateral conversion, to each extremal 
function of production parameters, corresponds an extremal 
function of control parameters. 

As agreed, the production quality index is an extremal 
function of its parameters. Therefore, function (1) in relation 
to the control parameters X l9 X 2 ,..., X n is also extremal. 


Adjustment-loss Time 

Assuming that a process having unchanged, fixed working 
components of control parameters X* 9 is under investiga¬ 
tion, and assuming also that, by the initial adjustment, it was 
possible, at some time t = / 0 , to attain the extremum value of 
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the production quality index, then under the influence of 
disturbance factors the production quality index will in time 
deteriorate spontaneously, in spite of the constancy of the 
control coordinates ( Figure 2). At the expiration of time 
the quality index will get out of the permissible limits. The 
disturbance effects are random functions of time or random 
values, although in some individual applications their mathe¬ 
matical expectations may dominate centred random elements. 

The change of production quality index with time Q ( t ) is 
also a random time function, known to be non-stationary for 
this process with a fixed adjustment. And so, repeating the 
above test, one gets new realizations Q (/) and new time values 
Ti (.Figure 2). 

The overall adjustment-loss time by the quality index Q 
is designated as the mathematical expectation M (Tj) of time 
intervals T So the overall adjustment-loss time expresses the 
mean value of time interval, after which the production quality 
index of the industrial process with a fixed adjustment gets out 
of the permissible limits. 



Figure 2. For determining the conception of general disadjustment time 


The adjustment-loss time, understandably, depends on the 
nature of the industrial process and its automation level by means 
of frequency automatic systems. If the overall adjustment-loss 
time is large, then a non-automatic, hand control is not 
difficult and there is no need to use a complex self-adjusting 
system. If the overall adjustment-loss time is small, then a 
person is unable to secure adjustment even with the presence 
of appropriate data transmitters and self-adjustment becomes 
necessary. 

It should be noted that the higher the speed of the industrial 
process and the stricter the demands on the quality of production, 
the smaller is the overall adjustment-loss time. Acceleration 
of the industrial processes and stepping up of demands on the 
quality of production are inherent characteristics of technical 
progress. Therefore, the application of self-adjusting control 
systems to industrial processes has a broad prospect. 


Equations of Extremal Control Processes 

It is assumed that, in the vicinity of the extremum point, 
serving as a work region of the system under consideration, 
the quality index ( 1 ) approximates with sufficient accuracy by 
the quadratic form of preceding values of control parameters 
and by the additional member SQ f expressing the influence of 
disturbing effects f l9 ..., f m : 


Q(0=e.+i t a^AXiit-rJAXjit-rJ + dQf 

z U = l 

a 2 e 

a ‘j ~ a ji ~ QXfiXj 

here 

AX V = X v - = X * + 5X V + <$X VVV 


(2) 


— X yl = AX V -f 8X V + SX VW 


are complete deviations of brought forth control coordinates 
(parameters), AX V * = X v * — X vi are working deviations 
of control coordinates, and Q e is the extremum value of the 
quality index. In case the computer of the quality index does 
not ensure smoothing (which is secured only by subsequent 
elements of the circuits) and the production parameter meas¬ 
uring instruments are practically non-inertial, or their inertness 
is accounted for in the values of time delays r i9 the output value 
of the computer equals: 


U(t) = Q(t) + 5Q n (t) 

Here SQ n ( t ) is the element conditioned by the errors of the 
production parameter meters and the errors of the computer. 
Thus 

tf<0=&+4- E ^jAX^t-^AXjit-x^+dQ (3) 

z i,j=i 

where 

8Q = SQ f + SQ„ 

The value U (J) in the multiplying elements of the synchronous 
detectors (correlators) is multiplied by the search signals SX V 
displaced in time by the delay simulator. The errors in delay 
simulation are designated <5r v . 

To the second entrances of the multiplying elements are 
transmitted values dX x {t — r v — dr v ) and the output signals of 
these elements equal V v = U ( t ) dX v (t — r v — <5r v ). 

The linear portion of the controlling system without any 
common restriction is divided into a set of filters and integrat¬ 
ing elements (Figure 1). The output working coordinates equal 

Xt=~ t W kv (D)V v 

U v= t 

Here || W kv (D)|| is the matrix of the transfer functions at low 
frequencies. Thus 

DX* k = DAXt + DX kl =t W kv (D)V, D=~ 

v=l aL 

or 

DAX*= X W kv (D) [u (t) 8X, (t - t v - <5 t v )] - DX kl 

V — 1 

utilizing equations (3) for U ( t ), one finds 

DAXt = \- X aiJ W kv (D) {[AXf (J- t ; ) 

L U j, V 

+ 5X iw (f-T £ )] x [AX* (t-xj) + 8X V (t - Xj)+8X jw (t- t/)] 

+ E W kv (O) L(Qi + 8Q)8X v (t — x— <5t v j]- DX kl (4) 

V 

(k=l,2,...,n) 

Summation by indices z, j, v is carried out within the limits 
from 1 to n. 
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ualitative Analysis of Extremal Control Processes with the remaining part of the input value. This is illustrated 

Quasi-stationary Regime by the graphs inF ‘Z“ re3 > showing a density spectrum curve S(m) 

. ^e input function, which is assumed to be stationary and 

The quality demand of an extremal control process reduces ergodic and an amplitude frequency characteristic A (co ) of an 

to the following. With considerable initial deviations from ideal high frequency filter. 

extremum the state point must move to the extremum as An ideal filter separates the high frequency constituent with 
smoothly as possible (without much overshoot). In a steady a spectral density Sd v (co) Figure 3(b) uncorrelated with the 

operation the state point must stay sufficiently close to the filtered component (spectral density Sw (co)), since the mutual 

extremum. spectral density of these components equals zero. 

Let eqn (4) be converted into: If the data meter controls the full input coordinate of 


DAX?;= £ a u W kv (D) 

*\ j, V 

[AXf (t - zd SXj (t - x ,) <5X V (t- t v - <5t v )] 

+ ~^W kv (p) 

^ V 

E a iA x 't (f- T ; ) AX* (t-Xj) 5X x (t -r v - <5 t v ) 

U j 

+ E atjWtoiD) 

i, j , v 

[AXf 0~xi) SX JW (t - x j) SX V (t t v — <5 t v )] 

+V.-DI. (5, 

S fk = Y E a u W kv (D) {[SX iw (t - T .) + 8X t (t - t ; )] 

j, v 

X l$ X jw (t — r j) + SXj (t - T ; )] SX V (t T y <5t v )} 

+ E ^v(£)[(a, + <50<$X v O-T v -5T v )] (6) 

Values ^/ 7c may be treated as the effect of errors, noises and 
search elements, brought to the outputs of filters of the syn¬ 
chronous detectors, provided there are no working deviations 
(AjQ* = 0). These functions do not depend on working devia¬ 
tions (it is assumed, that SQ does not depend on working 
deviations) and on the whole may only hinder the movement 
of the state point to the extremum. 

_ Thus, Sf k always play the role of disturbance effects and 
it is expedient to decrease them as much as possible. If the 
search elements SXj, have permanent constituents then, as seen 
from expression (6), it is impossible to decrease indefinitely 
Scp 7c by any increase of time constant filters of the synchronous 
detectors. Indeed, according to (6), the constant components 
SX V will cause deviations at the outputs of the synchronous 
detectors 

T E <*V WMIxMAX^+Y W kv (0)(Q kv +lQ)5X v 

J> v v 

where at least some of the transfer coefficients W ky> (0) are 
known to differ from zero, since otherwise the extremal con¬ 
trol circuit inefficient. Thus, it is expedient to secure zero 
parity of the permanent elements of search constituents i. e. 
the centring of the search oscillations. This is easily attained 
by installation of high frequency filters at the outputs of the 
search oscillation pickups. 

In particular, an ideal high frequency filter separates, from 
the input value, the high frequency constituent uncorrelated 


the system X v — X v * + dX v + dX VW) then the ideal high 
frequency filter in a stabilized operation separates the high 
frequency constituent SX V uncorrelated with the constituent 
+ ^ v r L should be noted that stationary X v * may be 
expected only in a stabilized regime of the system operation. 
In transient regimes X v * is a non-stationary random function 
and even with the use of ideal filters the search elements 
prove to be to some extent correlated with the working 
elements X v *. 



(b) 


Figure 3. Separation of an independent search component by an ideal 
high frequency filter 

However, as is seen from the following in the present system 
(perhaps even more than in other continuous extremal sys¬ 
tems), a quasi-stationary regime is profitable. In a quasi-station- 
ary regime the transient process times are large compared to 
correlated times of search elements. When a quasi-stationary 
condition is secured and with the application of high frequency 
filters near to ideal the search elements may be considered 
with a high degree of accuracy as uncorrelated with Z v *, both 
in a stabilized and in a transient condition of the system. 

Based on the above the search elements SX V is assumed 
to be centred by random functions as uncorrelated to AX*, SX W , SQ. 
Investigation of other members of the right portions of eqn ( 5 ) 
is now made. The second member of the right portion may be 
rewritten in the shape 

T^W kv (D)lF*dX v (t-c v -8x v )-] 

Z v 

where 

F* = YjAX? (t—Tj) AX* (t — Xj) (7) 
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In view of the definiteness of the signs of the functions of 
working deviations this member cannot facilitate the organiza¬ 
tion of movement to the extremum. 

Thus, members (7) play the part of impeding effects and it 
is expedient to reduce their influence to minimum values. 

The only accepted means of reducing the effects of these 
members is the raising of frequencies (decreasing correlation 
times) of the search elements at given times of transient 
processes of a closed loop or, inversely, increasing cumulative 
times at given correlation times of search elements. Either 
one or the other means result in a quasi-stationary regime. 
In a quasi-stationary regime the effects of members (7) can be 
neglected. The following members of eqns (5) 

E a ij Wiv (P) [AX* (r-T ; ) 8X Jw 0 1 - Xj) 8X V (t- T V - <5t„)](8) 

i, J, V 

although linearly depend on working deviations, are also playing 
the role of impeding effects. 

In fact, as agreed dX jw and SX V are uncorrelated and dX v 
are centred. Therefore, the mathematical expectations of the 
products 6X jw ( t — r 3 ) dX y (t — r v — Sr v ) equal zero. Thus, 
the expressions in the square brackets represent linear forms of 
working deviations, whose coefficients are centred ‘high fre¬ 
quency’ random time functions. These members can only 
increase the scattering of the trajectories of the state point 
during its movement to the extremum. 

In the quasi-stationary regime, because of the intensive 
suppression of the high frequency constituents, members ( 8 ) 
may be neglected. 

Turning to the investigation of the first member of the right 
portions of eqns (5) it is noticed that the product of search 
constituents may be represented as a sum of the mutual cor¬ 
related (at j 7 ^ v) or an auto-correlated function and a centred 
random function. Moreover, if the search constituents are 
stationary and are stationarily combined, then the correlation 
functions depend only on the argument difference. 

SXj (t - Xj) <5X V (t - T v - <St v ) = R Jv (r v - t,+< 5t v )+ (t ) 
where £ Sv (t) are centred random function members 

£ aiJ W kv (D)AX^ jv (t) (9) 

i, j, v 

play the same kind of negative role as members ( 8 ). In a quasi- 
stationary regime the influence of these members may be 
decreased to the same extent, as the influence of members ( 8 ), 
since the correlation times of function £ jv (t) are compared with 
the correlation times of function SX jw dX y . In a quasi-stationary 
regime one neglects the influence of members (9). 

Therefore, the general equations (5) are replaced by the 
following equations of a quasi-stationary regime of the system 
under consideration. 


Therefore, for a typical case, when the delay times Zj are not 
identical it may be assumed 


Rjviiv-tj+i 50 = 


o 

R v (Sx v ) 


for jVv 
for j = v 


(ii) 


It should be noted that the same correlations take place also at 
strictly identical delay times r v = z$, but with uncorrelated 
search constituents. In practice, uncorrelated natural search 
constituents may be obtained by means of installing instead 
of high frequency filters, band filters with non-overlapping pass¬ 
ing bands. The shortcoming of this method is the considerable 
lowering of the level or efficiency of the utilized search elements, 
especially in multi-instrument systems (n dimensional). 

Under condition (11) eqns (10) take the shape 


DAXt + E w ki (P) AX f(t- t,)= Sf k - X kl (12) 

i=l 

where 

W k m= E a tv R v (5x v )W kv (D) 

V=1 

It is also possible to introduce transfer functions of a closed- 
loop system, then 

AX * = A7m E (-1 ) k+v A kv mSf v -X vl ) (13) 

where 



D+W 1 c 1 (D)e~ Tib 

.. Wi„e~ Tnb 

A (D)= 



Ki(b)e~ T,b . 

D + W m (D)e~ T " b 


A 7 cv (D) is the determinant, obtained from A(Z>) by cancelling 
‘ K 9 column, V’ line. 

The roots of a characteristic equation 


a -(Ti+... + r n ) A 


2 e TlA + (A)... W‘ n (2) 


w: 


nl 


x^+w: n {x)\ 


= 0 (14) 


are simply determined in case when times r v are practically 
equal, the synchronous detector filters are neutral and possess 
identical transfer functions: 


K(K) ^v(D) = 


0 for v ^ fc 
W(D) for v = k 


In this case the characteristic equation (14) breaks up into n 
equations . tA 1 

4 (15) 


W(X) 


DAX*= E a, J RjA''v-*j+K)W kv (D)kXt(t-x i ) 

U J, o 

+ 8f k (t)-DX kl (10) 

(fe = l,2, ...,n) 

These general equations of a quasi-stationary regime are 
simplified in concrete, particular cases. 

First of all it is noted that the correlation times of search 
signals are small, due to the presence of high frequency filters. 


where C v is the semi-axis of the determining ellipsoid 


n 


E 


a ik AX ; AX fc =l 


If by decreasing the gain W (0) the roots of the characteristic 
equation are made so small, that e rX m 1, W (X) & W (0), then, 
in accordance with (15) 


A=- 


W(0) 

C v 2 


<0 
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and similar slow processes of extremal control always possess 
monotonous stability. However, with small gains the extremal 
control tune or the self-adjusting time is large and the errors 
considerable. 

Errors produced by drift of the extremum with constant 
speed equal 

AZ * =_ A^) v ? 1 ( ~ 1)i+VA ^( 0 )^ (16) 

It is noted that the value 

( __ i +v (Q) 

* A(0) 

equals the area bounded by the curved weighting function K kv (t) 
of a closed-loop system, corresponding to the transfer function 

A A v(i)) 

AID) 


v Ajj V (0) 


= K kv (t)dt=r kv 


Values T kv have time dimensions and will be called ‘areas of 
weighting functions’. If the system of extremal control is in 
general disconnected at AX k * ( 0 ) = 0, To W ki c = 0 df,= 0 
and ’ /c ’ 

AX%=- f‘d t=-X,.,t 


where we consider X u = const. 

With a disconnected system of extremum control, deviation 

a ^ 7 c V nCreaSes and in time exceeds the permissible value 
&Xjcg* where 

A X* g =-T k X kl (1 8) 


The increase in critical values of the gains and curtail¬ 
ment of times of transient processes of the extremal loop 
requires the diminution of transient lags r v between points of 
action of controlling elements and measuring points of output 
parameters in the industrial process itself. 

The control of output parameters may be realized at the 
output ofthe whole industrial process [Figure 4 (a)], at inter¬ 
mediate points [Figure 4 ( 6 )], and both at intermediate points 
and at the output [Figure 4 (c)]. 


To the computer 
♦ 4 l 




(c) 


Time intervals T k are called adjustment-loss times, as distinct 
from the general adjustment-loss time, mentioned above. 
From (16) and (17) it follows that 

A X* k = - £ T kv X vl 

V — 1 

Errors Ax k *, produced by constant drift of the extremum 
point in a closed system, naturally, must not exceed Ax k *. 

It follows from this that the weighting function areas* must 
satisfy correlations 

n 

T kv X vl < T k \X kl [ = \AX* g \ (19) 

Assuming in particular 

~ — A * _ i j =X k+kl = ... =JC nl =0 

(moreover AX vg * remain final values) one obtains 
\T kk \<T k 

i.e. the weighting function areas must be smaller than the 
adjustment-loss times. 

The curtailment of the weighting function areas (decrease 
of static errors) may be attained by increasing the amplification. 
However, the increase of gains, starting with certain values 
leads to the loss of stability of the extremal loop. 


Figure 4 

From the viewpoint of lag decreasing and possibility of time 
curtailment of transient processes, a circuit having parameter 
control at intermediate points has a decisive advantage over 
a scheme with control of final output [Figure 4 (a)] since it 
corresponds to the arrangement of information transmitters 
m the immediate vicinity of the controlling elements. However, 
this circuit also has one essential drawback. 

The quality index extremum, calculated on the basis of 
measurements at intermediate points, may not correspond to the 
extremum of output quality at the industrial process output. 

A more perfect circuit is the combined type [.Figure 4 (c)] 
where the control at intermediate points is combined with 
the output parameter control at the industrial process exit. 
In this circuit the signals of the parameter transmitters of the 
finished production pass through low frequency narrow-band 
filters Q, for instance integrating elements, after which they are 
added to the signals of corresponding transmitters, controlling 
the output parameters at intermediate points. This circuit 
conserves the quick action of circuits® and at the same 
time possesses the accuracy of control of slow changing para¬ 
meters, near to the accuracy of control of circuit 4 (a). 

However, the above extremal control system even with an 
improved informational section has a limited general application. 

In fact, as seen from eqn (12) the dynamics of quasi-station- 
ary processes of extremal control in this system depends on co- 
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efficients a iy of quadratic shape of the quality index, depends 
on time lag r v , errors of simulation of this delay dr v and 
intensity of search elements. 

For some industrial processes these parameters may be 
considered permanent, for others they are subject to com¬ 
paratively slow random variations. 

For the first processes the extremal control loop once 
adjusted maintains its efficiency for a long time. In the second 
case the extremal control loop itself needs periodic adjust¬ 
ment, accomplished by changing the transfer functions W kv (D) 
of the filters or just their gains W kv (0), and also, perhaps, 
the time delays. 

The necessity of adjustment arises due to the fact that, even 
though the stability of slow processes of the extremal control is 
maintained in a wide range of variations a iv , R v (<5r v ), a guarantee 
of the necessary quality of the extremal control is possible 
only by a suitable selection of transfer function filters. 

To this must be added, that even in those processes where 
parameters a iv , r v , R v (dr v ) remain unchanged the initial 
adjustment requires either a priori knowledge of these para¬ 
meters, or their experimental determination. 

An increase of the chance of general acceptance of extremal 
control systems with continuous industrial processes, in the 
sense of decrease of the necessary a priori information, may 
be attained at the expense of parametric extremum adjustment 
of the basic extremal loop. 

Self-Adjusting System with Parametric Extremum Adjustment of 
the Basic Loop 

To realize an extremum adjustment of the basic control, it is 
desirable to select the adjustment parameters of this loop and the 
quality index so that the latter shall be the only extremum in the 
working portion of possible variations of adjustment parameters. 


As an adjustment index of the basic loop, it is natural to 
select the mean value or, more accurately, the mathematical 
expectation of the same production quality index Q, which is 
utilized in the basic extremal loop. 

Moreover, in accordance with (2) 


M[Q] = e i +F X « y M[AX f (*-T I )A20(f-T J )] 

Z UJ= 1 

+ M [< 5 < 2 /] 


it is possible to show, if the errors of simulation lag are so 

restricted, that _ . A / i ^ . 

R v (5 t v )>0 (v = l,2, 


dll ••• din 


>0 


( 20 ) 


I d n t ...d nn \ 

where d uk = W u7c (0). 

Then the estimation M [Q] always has an extremum by the 
adjustment parameters d vJc whereupon this extremum is the 
only one in region (20). 

Taking into account the availability of the single extremum 
by the adjustment parameters and the general principle of 
extremal control, it is easy to lay out a control system of 
an industrial process with self-adjustment of the basic loop. 

This diagram is shown in Figure 5. The basic loop of the 
extremal control of the industrial process is here similar to the 
one previously examined. The difference is only in the presence 
of multiplier ‘matrix’ links, which realize varying transfer 
numbers d vk . 

Besides the basic loop the system has a loop of extremum 
adjustment of the transfer numbers matrix. 

It is assumed that the transfer numbers a iy of the indus- 
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trial process change slowly in time even as compared with 
quasi-stationary processes of the basic extremal loop. More 
accurately, it is assumed, that the time of substantial change 
of the transfer numbers a iv is considerably greater than the 
general adjustment-loss time T (see above). It is noted that, in 
principle, forced self-adjusting processes of transfer numbers 
are also possible. However, the dynamics of forced processes 
is complex, and for their organization it is not sufficient to only 
have the existence of an extremum of appraisal M [a] by the 
transfer numbers of the basic loop. Thus, as a typical regime of 
the system operation with two extremal loops, a regime with the 
following grading of process flow speeds is assumed: (a) Search 
elements in the basic loop (the frequency processes); 
(b) working processes in the basic loop; (c) search of os¬ 
cillations m the loop of extremal self-adjustment of the 
transmitting numbers, and (< d ) working processes of the 
extremal self-adjustment of transfer numbers (the frequency 
processes). 

With the above grading of process flow speeds, both the 
processes in the basic loop, as well as the processes in the self- 
adjusting loop of transfer numbers, are quasi-stationary. The 
dynamics of working processes, moreover, are near to the 
dynamics of ideal gradient systems (4). From this position and 
presence of an extremum of transfer numbers d vI , it follows 
that upon fulfilment of weak conditions (20) a quasi-stationary 
process of self-adjustment of the basic loop is always stable 

The above control system has considerable universal accept¬ 
ance By joining it with a plant (industrial process) with 
tew known characteristics, the system matches automatically 
transfer numbers corresponding to the quality index extremum 
m the framework of the given structure of the system. 

A further increase in ‘flexibility’ or universality of the system 
is made by introducing extremal adjustment of the delay simula¬ 
tor, extremal adjustment of the filter time constants and others 
However, all of this involves further complexity of the system. 


The Possibility of Extremal Control of Non-automated 

The main difficulties in introducing extremal control of 
industrial processes at the present stage are connected with the 


complete automation of output parameter control and complete 
automation of machine adjustment, securing the industrial 
process. The technology of most industrial processes even con¬ 
tinuous, did not yet reach the level at which it is possible 
to achieve continuous automatic control and adjustment. There¬ 
fore, it is of great interest to find the means of extremal con- 
trol for discrete semi-automatic or hand control and adjust¬ 
ment. 

The general algorithm of the extremal control and the com¬ 
puting section of the control system may, moreover, remain the 
same, as in a continuous automatic system. 

Estimation of the information output capacity of the 
measuring points, necessary for transmission of the search 
elements, indicates that for processes with considerable ad¬ 
justment-loss times a non-automated control is possible. For 
such processes, a periodic hand adjustment of machines is also 
possible, which guarantees the industrial process taking place. 
In this case the output signals of synchronous detectors are 
transmitted to the integrated indicators. Operators guided 
y the indicators of these devices, periodically correct the 
adjustment of the machines. With this type of organization 
of the extremal control the control system itself becomes a 
computer, either digital or analogue, equipped with input and 
output arrangements. The closing of the loop of an extremal 
control is here accomplished by human supervisorsand oper- 

crf-oro 
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DISCUSSION 


P. M. E. M VAN DER Grinten, Central Laboratory, Staatsmijnen, 
Geleen, Netherlands 

r i? sons y° u recommend the use of high-frequency search 
oscillahons. This implies that the phase-lag in the delay line applied 
11 increase and a small error m the time delay will cause large errors 
m the computations. Apart from this there will probably be dynamic 

amnlh ° ther than pure deIa y> which also affect the 

2 * “ fM h ‘ g u h frequencies. Might these complications have an 
adverse effect on the practical applications of the proposed method 1 ? 


A. A. Krasovski, in reply 

High-frequency oscillations are understood here to be those having a 
h.gh frequency as compared with the self-adaptiv eprocesses. As com¬ 
pared with a delay, the periods of these oscillations can be large For 
cases where this does not take place and the error of delay simulation 
is larger than the period (correlation time) of search oscillations, it is 
proposed to introduce self-adjustment for time delay of the model 
(simulator). The dynamic effects are taken into account in the system 
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Principle and Application of an Extremal Computer 
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Summary 

This paper deals with the structure of a special purpose computer for 
the automatic search of the optimum operating condition of a process 
consisting of one time constant and a delay. The structure proposed 
is independent of the dynamics of the system and it is not necessary 
to identify the process dynamics in order to apply this method. 

The first part of the paper is devoted to the theoretical considera¬ 
tions leading to the establishment of the proposed control law and its 
subsequent experimental verification utilizing an analogue computer. 
The second part of the paper describes the application of this method 
for the minimization of the ohmic loss of an alternator coupled to a 
power network. This experimental investigation permits confirmation 
of the results obtained from a theoretical study and helps to improve 
the design of the computer. 

In particular, this investigation demonstrates: (a) the possibility 
of building a simple computer for the search of an extremum using 
analogue and sequential circuits; ( b ) the self-adaptive properties of the 
computer even in the presence of fast perturbations and the stability 
of the method; (c) the possibility of compensating pure time delay 
(dead time) present in physical systems; and (d) the importance of 
filtering in the practical application of this method. 

Sommaire 

Cette etude propose une structure de calculateur destine a la recherche 
automatique d’un optimum de fonctionnement pour un processus 
doue d’une constante de temps et d’un retard. On doit noter que cette 
structure est independante de la nature de la dynamique du systeme 
commande et que Fapplication de cette mdthode n’implique pas la 
determination de cette dynamique. 

Une premiere partie est consacree a la mise au point et a l’experi- 
mentation sur un calculateur analogique de la loi de commande 
proposde. Une deuxieme partie decrit Fapplication de cette methode 
& la minimalisation des pertes ohmiques dans un alternateur couple au 
reseau. Cette etude experimentale a permis, d’une part de confirmer 
l’etude theorique, d’autre part d’apporter des perfectionnements 
pratiques au calculateur etudid. 

Cette dtude montre en particulier: (a) qu’il est possible de rdaliser 
selon des structures simples, analogique et sdquentielle, un calculateur 
de recherche d’extremum; ( b ) le caractere d’auto-adaptation, en 
particulier vis-4-vis de perturbations rapides, et de haute stability de la 
methode; (c) la possibility de compensation des retards propres a 
tout systeme physique; et ( d ) l’importance du probleme de filtrage 
pour la mise en application du procede. 

Zusammenfassung 

In der vorliegenden Untersuchung wird der Aufbau eines speziellen 
Rechengerates diskutiert, das automatisch das Optimum eines Pro- 
zesses sucht, der sowohl eine Verzogerung l.Ordnung als auch eine 
reine Totzeit besitzt. Hervorzuheben ist, daB der Aufbau des Rechners 
unabhangig von der Art des dynamischen Verhaltens des geregelten 
Prozesses ist; zur Anwendung der vorgeschlagenen Methode ist es 
nicht notwendig das dynamische Verhalten des Prozesses zu ermitteln. 

Der erste Teil der Untersuchung schildert die Herleitung des vor¬ 
geschlagenen Optimierungsgesetzes sowie experimentelle Untersuchun- 
gen mit Hilfe eines Analogrechners. Im zweiten Teil wird die An¬ 
wendung der Methode zur Minimisierung der ohmschen Verluste in 
einem mit dem Netz gekoppelten Generator beschrieben. Die hier 


gewonnenen experimentellen Ergebnisse haben einerseits die theore- 
tischen Untersuchungen bestatigt, andererseits praktische Hinweise 
fur den Bau des Rechengerates geliefert. 

Die vorliegende Untersuchung zeigt unter anderem: a) Es ist mit 
einfachen analogen und sequentiellen Schaltungen moglich, ein 
Rechengerat zu bauen, das selbsttatig die Extremwerte sucht. b) Die 
Eigenschaft der Selbsteinstellung des Gerates, auch bei Vorhandensein 
schnell veranderlicher StorgroBen, sowie die gute Stabilitat des Ver- 
fahrens. c) Die Moglichkeit, die in physikalischen Systemen vor- 
handene Verzogerung (Totzeit) auszugleichen. d) Die Bedeutung des 
Filterproblemes bei der praktischen Anwendung der Methode. 


Introduction 

Many industrial processes have an optimum working region 
which can be expressed as an extremal value of a figure of merit 
/ Pic)- This figure of merit depends upon the acting variables 
Xi on the one hand and the disturbing values p k on the other; the 
latter coming either from variations in the system environment 
or from the system itself. These disturbances act slowly with 
respect to the dynamics of the system and are likely to modify 
the state of the system very appreciably. In the present study a 
method is established of controlling the acting variables from a 
measure of the figure of merit. The function of this closed-loop 
control is to compensate for the action of the disturbing values 
which tend to shift the system from its optimum working condi¬ 
tion. Moreover, this servo should be able to seek the optimum 
from any initial conditions. In order to obtain this double ob¬ 
jective, measurements are made in the process which allow the 
figure of merit to be determined more or less directly, and acting 
elements to modify the process are installed. In general the dis¬ 
turbing values are inaccessible, and all that is known about the 
function / is that it has an extremum. This function is usually 
only available after a dead time or time constant which are in¬ 
herent in all physical systems. This means in fact that only the 
function h (x h p k ) is available, connected to / (x h p k ) by a differ¬ 
ential relationship. 

The following remarks can be made about this type of 
closed-loop system. The study of a system whose performance is 
determined by several independent acting variables can be con¬ 
sidered as a system with one variable if at any given instant only 
one variable is controlled. The control is switched successively 
from one variable to another. Under these conditions it is neces¬ 
sary only to study a system with one acting variable, which is 
designated x. It is necessary to modify the acting variable x with 
a speed dx/dt whose sign and amplitude are determined, x itself 
cannot, in fact, be controlled as in a normal servo system since 
the reference value which corresponds to the extremum is un¬ 
known. This remark makes apparent the adaptive character of 
the control envisaged, since the role of this control is to make the 
system tend towards an optimum which is unknown and possibly 
variable. 
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This study has led to consideration of certain special cases 
which are likely to complete the work undertaken in the same 
spirit by other workers 1 ~ 8 . The study has been done theoretically 
by analogue methods and then experimentally on an alternator 
connected to a supply. 


Analogue Study 

First, the system to be studied is defined from a static and 
dynamic point of view; the method of control is then envisaged, 
particularly the nature of the necessary switching. The study of 
limit cycles enables definition of ideas of stability of the optimum. 
Finally, the influence of the various control parameters is 
studied. 

Description of the System 

The system studied is represented schematically in a simplified 
form in Figure 1 . The controlled system comprises the elements 
B and T, where B represents the function to be optimized, here 
supposed to be a minimum. Near the minimum the function 
fix) can be considered as xx 2 (oc > 0) taking the origin as the 
abscissa of the minimum. The element T represents a time con¬ 
stant r which can be due either to the system itself or to the 
measuring device. The integrator A does not, strictly, belong to 
the system but to the controller; but it can be considered as in¬ 
herent in all control devices, and it is convenient to connect it 
artificially to the system. The element R represents a pure dead 
time r r . It is possible to lump at the input the sum of the lags 
which appear at any point in the chain. These lags intervene then 
simply as a retarded action. 

The effect of a time constant r e at the input can easily be 
compensated since there is the derivative of the acting variable. 
It is only necessary to replace the integrator with an element of 
transfer function (r e + 1/p). Hence the general system to be 
considered is that shown in Figure 2, which is easily derived from 
Figure 1. Finally, a constant amplitude K is imposed on the 
derivative of the acting variable. 

The object of the control is to determine the sign of dx/dt 
knowing h [x(Y)]. The relationships defining the dynamic func¬ 
tioning of the system can be written 


The initial conditions fix the values x(0) and h [x(Q)]. Con¬ 
sidering x as the independent variable and introducing the di¬ 
mensionless variables defined by 



(3) 


u_ h 


(4) 


the solutions for the system are 


from which 


and 


X 2 


r=—- £ x+l+Ae- £ * 

(5) 

dH v , 

s-sle x 

(6) 

*Z- 1+X9 -* 

dX 2 ~ L+A 

(7) 

rr X 2 dH 

(8) 

H X ' .y.d’H 

(9) 


A is a dimensionless coefficient defined by the initial conditions. 

Consider the evolution of the system in the H(x) plane. This 
representation is useful since eqn (2) implies that X is a linear 
function of time. The dynamics of the system are thus represented 
in this plane by two networks of curves, symmetrical about the 
vertical axis, which correspond to the two possible values of e. 
These curves are represented in Figure 3 for the case s = — 1. 
Also drawn in Figure 3 are 


(a) 


dH ^ X 2 

S- 0 ’ h= ~T 


from eqn (8) 


this is a parabola representing the static state F 


'z-^— + h = (xx 
at 


( 1 ) (b) 



y2 

or H=~^eX 


from eqn (9) 


dx 
d t 


=sK (s— +1) 


(2) T hese are two parabola H symmetrical with respect to the 
vertical axis; the parabola relative to e = — 1 is designated H ~. 



Figure L Block diagram of the system 



Figure 2. Equivalent diagram 
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These curves also correspond to the zero values of dh/dt 
and d 2 h/dt 2 . 

The trajectories are asymptotic to the symmetrical parabolas 
of equation y 2 

H =±--eX +1 (10) 

Switching the acting variable causes the point representing 
the dynamic state of the system to pass from one network of 
curves to the other. 


Switching 

Proposed Law —The proposed method of searching for the 
minimum consists in acting at constant speed; positive or nega¬ 
tive, on the acting variable x according to eqn (2); that is, to 
impose a positive or negative sign on the derivative dx/dt. The 
choice of sign is determined by a switching device which 
operates according to the following sequence. 

The sign of s is modified each time that the quantity 



d 2 h 


d? 

or 

G = 

with 

A = 


-<5 = g 



S 

2ccK 2 


passes from a negative to a positive value (switching N or 
‘natural’), and this switching is to continue with a period 
r w (W = r Jr) as long as the function G remains positive 
(switching F or ‘forced’). 

<5 is a positive or negative threshold. A zero value of S with 
the proposed switching law leads to a search for the minimum 
value of dh/dt; in other words, to make h tend to its minimum 
value as quickly as possible. 



Figure 4. Block diagram of the closed-loop system 



Figure 5. Switching processes with a threshold and lag 


The forced switching has been introduced both to take 
account of part of the unfavourable initial conditions which do 
not allow the system to switch naturally, and to avoid instability 
in the case where, upon switching, the system diverges (G > 0). 

The switching process can now be defined for a system with 
dead time (Figure 4). C represents the switching device which 
produce the law defined above. 

Consider (Figure 5) the case of one curve (e = — 1). The 
quantity ^ 2 ^ 

which should start the natural switching action becomes zero 
at S when < • < f rom e qn (7) (11) 


1 + Ac 
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Switching is only effective on dx/dt at a time r r later. There 
results the relationship between the abscissa X rs of actual 
switching and of starting 

X rs ~X s = sR (12) 

R is the dimensionless lag x r jx. 

Lastly the point P , being on the considered trajectory, must 
satisfy eqn (5) 

X 2 

H rs =--^-sX rs +l+le~ sXrs (13) 

The elimination of X and X s between eqns (11) to (13) leads 
to the equation for the actual switching points 

H rs =^-sX rs +l-(l-A)e~ R (14) 


Properties 

L The switching points H rs described by eqn (14) are para¬ 
bolas. When A or R vary, these parabolas keep the same axis 
of symmetry X = e. They are derived from parabolas H by 
a vertical shift of 1 - (1 - A) t~ R . If there is no dead time, 
natural switching occurs on parabolas H s whose equation is 

H s =~—eX s + A (15) 

If there is no threshold or dead time, natural switching 
occurs on parabola H given by 

H=^-sX s (16) 


-• when R is different from zero, the switching on H rs 
occurs only if switching action has started on H s beforehand. 
If not curve H rs can be crossed without natural switching [Fig- 
ures 12 (c) and 13 (c)]. 

5. A threshold A equal to unity is equivalent to an infini te 
dead tune. The switching parabolas H rs and H s tend to the 
asymptote parabola X, switching cannot occur, and the system 
diverges. Hence the condition A < 1 must be observed. 


4. The switching mechanism described is useful only if the 
period of forced switching W is greater than the lag R. In fact 
tihe control principle itself implies the observation of the sign of 
G. In the presence of a dead time, the start of natural switching 
action causes, in principle, the start of a forced switching action 
which will occur in all cases where W is less than R. Hence the 
condition W > R must be observed to avoid false switching. 

5. The two parabolas H s divide the plane into four regions 
(.Figure 6). In zone I, G is negative; no switching can occur 
m this zone. The dynamic curves of the two networks are in¬ 
creasing, and all cross a parabola H rs on which a switching 
takes place [Figure 15 (e)]. 

In zone II, G is positive; hence the production of periodic 

forced switching as long as the system remains inside this zone 
{Figure 15 (</)]. 


All curves of the two networks are decreasing and finally 
enter the zones III and IV [Figure 15 (c)]. The final state should 
establish itself, at least partially, in the zones III and IV where 


natural or forced switching leading to limit cycles can occur, 
depending on the initial conditions. 

6. Eqns (14) and (16) suggest the possibility of compen¬ 
sating for a dead time by a negative switching threshold, ff the 
equation for compensation is produced 

1—(1—zOe _ * = 0 
or 

F=l (17) 

where 

P = (1-A')q~ r 

the switching curve H rs becomes the curve H of switching in the 
absence of threshold A and dead time R (Figure 5). This remark¬ 
able result is verified on the analogue computer [Figures 12 (e) 
and 13 (e)]. The sum of these properties justifies the proposed 
switching law. 

Limit Cycles 

The application of constant, non-zero, positive or negative 
acting variable leads, in the absence of disturbances, to stable 
periodic oscillations. Two principal types of limit cycle can be 
observed in the plane H (X ): 

type NN where only natural switching occurs, 

type NF where natural and forced switching alternate. 

Figure 7 shows these two types of cycle in the presence of 
dead time and a threshold. 



Figure 6. Switching zones 



Figure 7. Limit cycles 



PRINCIPLE AND APPLICATION OF AN EXTREMAL COMPUTER 


Calculation of Limit Cycles NF —Calling A 0 and the two 
values of A relative to the two trajectories which constitute the 
limit cycle, the following equations are obtained. 

2 X m =A t e ~ Xm - X 0 e* M [from eqn (13)] 

2X M =X 1 e~ XM -X 0 e XM [from eqn (13)] 

\+A 0 e x * = A [from eqn (11)] (18) 


X-X m = R 

x M -x m =w 

X s corresponds to the crossing of H s . 
Values can then be deduced for 


Me 2Tr - 1J-2W^ 
1 2 (e w -1) 

. fi(c 2W -l)-2W 
M ~ 2(t w —V) 


(19) 

( 20 ) 


Figure 8 shows the variation of X m as a function of W and 
Calculation of Limit Cycles NN— Similar equations to 
eqn (18) show that in this case 


X M =-X m (21) 

X M =-A(l-e 2X M) (22) 


The cycle is symmetrical, and its amplitude is given by 
solution of eqn (22). The amplitude is independent of W, 
Figure 9 showing how it depends upon ji. 

Limiting Case —The curves of Figures 8 and 9 can be used only 
if the type of limit cycle considered is known beforehand. The 



limiting case which separates cycles NN and NF is that where 
the point X m lies on H $ ; in other words, when the following 
equation must be added to those of eqns (18) 

1 + A 1 e“ Xm = d (23) 

From these can be deduced ecjn (24) which exists between 
the parameters R, A and W in order that the limit of the appear¬ 
ance of a cycle NN may be reached 


-R = log 


(L£KM 

L{\-q- w ){1-A)-2W. 


(24) 


This expression leads to the plotting of the network of 
curves of Figure 10. 
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Figure 11. Limiting cases of the occurrence of limit cycles NN and NF 


Three zones are indicated in Figure 11. Proper operation 

1S ^ 0t ?° S T Slble ln zone 1 since R > w - Zone III leads to limit 
cycles NN or NF, depending on the initial conditions, and in 
zone II all limit cycles are NN. In zone II the interest of oscil¬ 
lations NN is that use can be made of their symmetry to deter¬ 
mine the optimum state of the system and to fix it in this 

position. Such a realization is described in the experimental 
section. 


Influence of the Parameters 

The Nature of the Parameters —Two sorts of parameter can 
be distinguished: first, those connected with the system and which 
cannot be acted on. These are the time constant r and the dead 
time r r . The time constant r occurs as a normalized value by 
means of the product Kx, while the dead time is defined by the 
dimensionless variable R proportional to the real dead time of 
the system. 

The second type of parameter is connected with the controller 
and can be acted on. The dimensionless switching time W which 
is proportional to the real period r w , the gain K, considered as 

Lo Z, and the dimensionless threshold A with value 
0/(2 oc K 2 ). 

In practice A and R intervene via the parameter 
jU=(l — zl)e~ s 


Two identical values of fi correspond, all other things beini 
nS % n W ° ldentlcal s y ste m behaviours [Figures 12 (e) ant 
7 Property 2 is satisfied. Figure 13 (c), to be comparec 

Wl h f 12 illustrates a case where this condition is 

not fulfilled. 


The influence of these different parameters has been shown 
on the analogue computer. The recordings of Figures 12 to 15 
where the numerical values are given in dimensionless form, lead 
to the following conclusions. 



Results 

Oscillations AW—These oscillations are symmetrical w 
respect to the optimum [Figure 12(a)]. Their amplitude 



Figure 12. Influence of the parameters on the behaviour of the system 
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Figure IS. Influence of the parameters on the behaviour of the system 


independent of W [Figure 12 (b) and (c)], proportional to K 
[Figures 12 ( b) and 13 (b)] and reduces when fx increases [Figure 

12 (a) and Qb )]. 

Oscillations NF —The average abscissa of these oscillations 
is further away from the optimum as fx and W increase [Figures 

13 (d) and (e); 12 (d) and (<?)]. The amplitude of these oscilla¬ 
tions is independent of / x [Figure 13 id) and (e)] } proportional 
to W [Figure 12 ( d ) and (e)] and to the gain K [Figures 12 (b) 
and 13 (b)]. 

Settling Time —The settling time is proportional to the time 
constant r. An increase in gain reduces slightly the settling 
time [Figure 14 ( a) and (b)]. In practice this time is of the order 
of magnitude of the time constant r of the system. 

Accuracy —The accuracy is defined by the average ordinate 
of the limit cycle; that is, by the amplitude of cycles NN [Figures 
12 (b) and 1.3(b)] or the average abscissa of cycles NF [Figure 
12 (d) and (e)]. 



Figure 14. Influence of the parameters on the behaviour of the system 
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Figure 15. Influence of the parameters on the behaviour of the system 


Choice of Parameters 

The forced switching period is made as small as possible in 
order to reduce the settling time of the system when it follows 
an unfavourable trajectory, and afterwards to allow the quickest 
possible compensation for disturbances. In the absence of a 
dead time, this period can be of the order of a twentieth of the 
system time constant r (W = 0-05). With a dead time, W is 
chosen to be slightly greater than this dead time. 

The gain must be as high as possible to reduce the settling 
time and to minimize the effect of disturbances. Its upper limit 
is set by the amplitude of the oscillations. 

The threshold is chosen {Figure 16) to compensate for the 
lag (p, = 1). In practice the threshold will thus be negative, 
which assumes the condition A < 1 whatever the value of «, 
which defines the unknown static curve. 


Experimental Study 

Statement of the Problem 

The pioposed method has been tested experimentally on an 
alternator connected to a mains supply with the object of 
optimizing the reactive power produced. It is known that for 
a given active power, the reactive power, and consequently the 
current produced, passes in the static state, through a min imum 
for a given value of excitation current. This minimum is displaced 
when the active power varies as is shown in Figure 18 (a). 

This point corresponds to the minimum of the ohmic losses 
in the alternator, and can be considered as a point of optimum 
operation. The non-linear characteristics are not known before¬ 
hand. The main disturbances are the variations of active power. 
Dynamically there is a preponderant time constant (5 sec) 
introduced by the detection of the current supplied by the 
alternator. Hence the dynamics of the alternator itself do not 
intervene to determine the order of the system. The dead time 
also is negligible. 

Brief Description of the Experimental System 

The block diagram is shown in Figure 17. The alternator 1 
used has a power of 6 kVA and is driven by a Ward-Leonard 
group which supplies the active power. The detection 2 of the 
current supplied by the alternator is made on the three phases 
by current transformers. The excitation of the alternator is 



Figure 16. Choice of parameters W„ and A 0 for a lag B 0 
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regulated by two silicon controlled rectifiers 14 whose gates are 
connected to the output S of the optimizing computer. The time 
constant of the excitation, of the order of 0*5 sec, can be com¬ 
pensated for by using the fact that the computer imposes the 
derivative dx/dt of the acting variable. 

Operations Performed by the Computer—Results Obtained 

The computer used for the optimal control of the alternator 
reactive power comprises analogue elements (operational d.c. 
amplifiers) to produce the algebraic and differential operations, 
and sequential static circuits for the logic operations. 

This computer is designed first to demonstrate the principles 
of the method proposed in the theoretical part, and secondly 
to bring certain improvements, especially to suppress the 
oscillations around the optimum working point. 

Analogue Elements —They realize the following functions: 

(1) First and second derivatives. These are produced by two 
operational amplifiers 4 and 5 (.Figure 17) which ensure at the 
same time a suitable filtering for the signal. 



Figure 17. Extremal control of the reactive power of an alternator con¬ 
nected to a supply. Block diagram of the installation 


1 Alternator 
3 Filter 

5 Second derivative 
7 Action law 
9 Sign inverter 
11 Integrator 
13 Switch 


2 Current detector 
4 First derivative 

6 Control of the opening and closing o 
the optimizing loop 
8 Switching law 
10 Switch 
12 Memory 
14 Excitation control 


(2) Constant action law. The behaviour of the system in the 
case of an action law of constant amplitude has confirmed the 
practical validity of the theoretical study, despite the differences 
between the real system and the ideal system studied, especially: 
the presence of noise, the non-parabolic nature of the static 
curve, and a more complicated dynamic system. 

(3) Proportional action law. The behaviour of the first and 
second derivatives [Figures 14 (c)] during the search for the 
optimum shows that an action law dependent upon the amplitude 
of the second derivative allows the amplitude of the oscillations 
to be reduced, and at the same time reduces the settling time. 
The effect of disturbances is hence also much reduced. The action 
law given by the element 7 has the form shown in Figure 19 (a), 
and the adjustment of the parameters k, K, S ± and S 2 allows the 
best action law to be chosen for each case. 


(4) Average optimum value. The theoretical study showed, on 
the hypothesis of a parabolic approximation, the symmetry of 
the oscillations of type NN around the optimum. This property is 
very useful since it becomes possible to use these oscillations to 
derive the average value of the optimum and to impose this 
value as a control value on the acting variable x. The oscilla¬ 
tions are then eliminated [Figure 18 (/>)]. The presence of internal 
or external disturbances, however, necessitates the elaboration 
in a systematic manner of new optimum values of the acting 
variable by closing the control loop. This function is produced 
by the logic elements in the computer. 

Logic elements —The logic elements perform the following 
functions (Figure 17). 

(1) Sign of the action law. The sign of the action law (dx/dt) 9 
must be produced according to the proposed switching law. 
This switching law requires a sequential circuit 8 which does 
not pose any special problem. This circuit has been produced 
using semiconductors. 

(2) Memorization of the average optimum value. The average 
optimum value is stored in an analogue memory (12) until the 
production of the next average optimum. The acting variable x 
can be connected by means of the static switch (13) either to the 
integrator (11) comprising the last stage of the optimizing com¬ 
puter which produces x from dx/dt (in this case the optimization 
loop is closed) or to the memory (12) after a given number of 
switchings. In this case the optimization loop is open; the 
acting variable x is regulated to its average optimum value. 

(3) Control of the opening and closing of the optimization loop. 
The element (10) which opens and closes the optimization loop is 
controlled by the logic circuit (6) in order to produce the follow¬ 
ing functions. 

(a) Opening of the control loop (connection of x to the 
analogue memory) when the average optimum value is reached. 

(b) Closing of the control loop when the second derivative 
exceeds a predetermined value, i.e. in the presence of sufficiently 
rapid disturbances. 

(c) Closing of the control loop at predetermined time inter¬ 
vals in order to compensate for disturbances which are too slow 
for their second derivatives to exceed the fixed threshold. 
Figures 19 (b) and 18 (b) illustrate these different functions: 

The approach to the optimum (ABC), the production of the 
average optimum value (BC), the opening of the control loop 
and the connection of the acting variable to this average optimum 
value (point C), and the trend of h (t) towards its correspond¬ 
ing optimum value (CD). The disturbance appearing at point D 
[Figure 19 (b)] automatically causes the closing of the control 
loop and starts a new search cycle (DEFG). Finally, several 
search cycles from different initial conditions are indicated; 
compare Figures 20 and 21 with Figure 15 (c), (d) and (e). 

It can be finally stated that, according to this conception, the 
optimizing computer produces the optimum control value, 
imposes it on the acting variable x, and modifies it in order to 
take account of fast or slow disturbances which tend either to 
displace the system from its optimum working point or to change 
the optimum value itself. 

Conclusions 

The interesting points of the proposed method, the problems 
encountered and the directions of future development are now 
considered. 


535 











r 

for a given amplitude of oscillations. The settling time can be j 

said to be of the order of the system time constant. ■ 

Notice also the inherent stability of this method of optimiza- \ 

tton An unwanted spurious switching, which places the state . 

of the system on a divergent trajectory, is corrected by a forced I 

switchmg an instant W later. The same can be said for the ap¬ 
pearance of very fast disturbances. :. 

One of the main problems of the proposed method is the pro- r 

duction of the first and second derivatives; whatever method is i 

envisaged, analogue, sampling, etc., there remains the problem 
of separating the useful signal from the noise. This separation 
becomes easier as the noise and signal have different frequency 
spectra. Notice that the problem is much simplified if only the 
sign of the second derivative is required. 


From a general point of view, methods of automatically 
searching for the optimum working point have the advantage 
that they can be applied without necessarily knowing a great 
deal about the dynamic behaviour of the system, which is neces¬ 
sary in methods which require a mathematical model of the 
system. 

So far as the method itself is concerned, the choice of the sign 
o a quantity dependent upon the second derivative as a switch¬ 
ing criterion has allowed the use of forced switching periods 
w ich are very small compared to the system time constant. This 
is a great advantage since the range of disturbances which the 
system can tolerate is proportionately increased. 

Also the fact of being able to reduce the forced switching tim e 
a ows an increase in gain, and hence a reduction in settling tim e. 
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(a) 

(a) Kr min = 1 
W = 004 



(b) K T m j n — 1 

^" T max~ 6 

IF = 004 




<- - > 

(a) 

(a) Kr m i n — 1 
~ ^ 

fF = 0-04 


Figure 21 


Preliminary tests carried out on the analogue computer 
showed that this method could be extended to systems with 
several variables and of higher order. Finally, the proposed 
method is in the process of application to a chemical reactor, 
working in the gaseous phase, in order to obtain at all times the 
maximum efficiency. 

This study was carried out at the Battelle Memorial Institute, 
Geneva, to which the authors extend their thanks. They would also 
like to thank Mr. R. Bertoldi for his invaluable collaboration. 


Nomenclature 

/ Figure of merit (cue 2 ) 

G Dimensionless value of g ( d 2 H/dT 2 — A) 
g Switching function (d 2 /z/dr 2 ) — d (sec -2 ) 

H Dimensionless value of h ( h/2 ocKh 2 ) 
h Dynamic value of figure of merit 
K Constant amplitude of the derivative dx/dt (sec -1 ) 
k Minimum amplitude of the action law (sec -1 ) 
p Laplace operator (sec -1 ) 
p k Disturbance 
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R Dimensionless value of the dead time (r r /r) 

T Dimensionless time (t/r) 
t Time (sec) 

W Dimensionless switching period (r w /r) 

X Dimensionless value of x (x/Kt) 
x Acting variable 
a Coefficient of the parabola law 
A Dimensionless value of threshold (S/2 aK 2 ) 

S Switching threshold (sec~ 2 ) 

e — i 1 

A Integration constant 
F Coefficient (1 — A) er R 
t System time constant (sec) 
r e Time constant of the acting element (sec) 
r r Dead time (sec) 
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DISCUSSION 


M. Hamza, ETH, Zurich, Switzerland 

Extremum seeking regulators have a wide and important field of 
application and several methods have been suggested for their 
design 1 ” 3 . The system suggested by Perret and Rouxel is novel, 
especially their method of employing the second derivative, and they 
are to be highly congratulated for their excellent work. A technique 
will be presented which does not possess some of the disadvantages 
of the system suggested in this paper, namely: 

(1) Its performance is dependent on initial conditions (Figure A). 

(2) Time is required to build the average. Figure 20(b) of the paper. 

(3) If the non-linearity is not symmetrical, the method of building 
the average will not lead to extremum. 

The following technique was derived for higher-order systems 
having various types of non-linearities and for systems having n vari¬ 
ables. A paper describing this technique in full, with several illustra¬ 
tions, will be published soon. The method is based on determining the 
slope of the non-linear function by measuring the discontinuity in an 
appropriate derivative of the system’s output when the input is (for 
convenience) a step. In some respects, this method is similar to the 
sinusoidal perturbation technique 3 and possesses its advantages. As 
an illustration, with reference to Figure A and assuming no delay to 
be present and that the integrator is not a perfect one, this may be 


given by 1/^ + 1, the discontinuity in the second derivative of the 
system’s output with respect to time when the system input is a step of 
magnitude A may be easily seen to be 


d2 *(0 XA 1 jr* 1 , „ 

dt 2 p (t^ + 1) xp + \ P p 


( 1 ) 


where 


AK* 

T t T 




*o refers to x: at / = 0. Note that 


If/= ocx 2 , then 


. d 2 *(0*) AK* 


sign 


dt 


2—=sign- 


TiT 


d 2 h 


AK* 

dt 2 


TjiT 


K = 2 ax 


( 2 ) 

(3) 

(4) 

(5) 

( 6 ) 
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and d 2 3 fe(0*) _2^g 

dt 2 TjT X 

Thus a simple logic circuit governed by eqn (4) or by eqns (4) and 
(5)—where eqns (4) and (5) are first used to estimate the position of 
the extremum, since the discontinuity is proportional to x, and once 
near the extremum eqn (4) alone—may be used to govern the behav¬ 
iour of the controller. Near the extremum the oscillations may be 
reduced using prediction linearization and other techniques as sug¬ 
gested in 1 * 2 . If a delay is present it may be determined by measuring 
the difference in time between the application of the input step—or 
disturbance—and the occurrence of the discontinuity in the system 
output. Thus the effect of the delay may be taken into account. 

In the original version of Perret and Rouxel’s paper, which was 
written in French, they showed that the second derivative suffered a 
discontinuity proportional to X , and Nour Eldin (in a colloquium) 
pointed out that for the case studied by the latter authors the dis¬ 
continuity may be used to identify the extremum. He obtained this 
result independent of the author and his method was based on a 
phase plane study. 



I would like to point out that the results in Figure 15 (a) and ( b ) 
might be slightly misleading since Kr min and Zr max in both cases are 
not the same. Further, how could the first switching in Figure 15 ( e ) be 
explained ? It does not occur on a parabola and it must be of the N 
type. Finally, it is worth noting that although the shortest time to 
reach the extremum from A , Figure B, is ABC, the shortest from point 
4 A' is not A'B'C but could be A'B"C, as may be readily proved. 
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this point its acceleration is positive. There will be a jump in the 
acceleration at the switching point a. The equation of this jump is: 

jump amplitude = f (a 1 T)x 

where oc is the order of the parabola and T the time constant of the 
system. If the plant characteristic does not vary during this time the 
jump amplitude will be directly proportional to the distance from the 
minimum. The direction of the jump depends on the transition direc¬ 
tion of the control variable U. These jumps are shown in Figure A. If 
we use this jump property for detection of the distance from the 
minimum, we choose a certain perturbation sequence, say U~ to U + . 
Measuring the jump in the second derivative will give the distance 
from the minimum. For varying plant characteristics one can identify 
the time constant and calculate the distance from the minimum. 





Jump 




1 Jump 

|iL 






Transition 


0 □ Transition E— 0 + 

u + —- If u“—u 

Jump - P (aj)-x 


Figure A 



H. Nour Eldin, Swiss Federal Institute of Technology, ETH, Zurich , 

Switzerland 

The paper presented by Perret and Rouxel is interesting as it uses the 
second derivative for control law. Using the second derivative has its 
drawbacks and I shall demonstrate an inherent property in this hill 
climbing problem. 

The dynamic characteristics of the system can be shown in the two 
planes U + and U~. At point a + the control variable U is negative and 
crosses the zero acceleration point. If the control variable is switched 
to U + at this point, the point a + will be point a~ in the £/ + plane. At 
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The jump property can be used for one-dimensional and also for 
multi-dimensional systems and it can be extended to multi-valley 
problems. It works also in the presence of noise (by averaging before 
and after transition). In the presence of delay time the system can work 




with modification for delay estimation. For a plant of higher order, 
Mr. Hamza at the Swiss Federal Institute of Technology, working inde¬ 
pendently, has proved the jump property for the higher-order system. 
One can build a hill climbing system in a very simple manner using a 
logic circuit which determines the sign of the jump. This system does 
not require any plant identification. It measures the second derivative 
before and after the transition and determines whether the plant is at 
the right or at the left from the minimum. The block diagram and the 
results are shown in Figures B to E. The system goes to the minimum 
with good accuracy. May I remark that reaching the minimum point 
m the minimum time can be achieved by switching off the system at 
X = 0 (or oscillations around X — 0). 

R. Perret and R. Rouxel, in reply 

The improvement prepared by Hamza and by Nour Eldin presents 
some advantages for first-order systems, especially for dynamic initial 
conditions located in zone 1 of Figure 6 of our paper. Nevertheless, 
we would like to make the following remarks: 

(1) For Tzth order systems, this method implies the elaboration of 
the (n + 1) th time derivative which is a severe practical limitation. 

(2) This method needs a periodic test signal superimposed on the 
actuating one. The dead time inherent to physical systems implies that 
between every test input and test output signal a time delay at least 
equal to the dead time is required which seems to be difficult to com¬ 
pensate. The method proposed by the authors avoids these dis¬ 
advantages. The last two questions of M. Hamza can be answered as 
follows. 

(1) In Figure 15(b) of the paper, the maximum value of the gain is 
higher than the gain related to Figure 15 (a). For a given action law, 
the amplitude of the oscillations usually increases with the gain. In this 
specific case [Figure 15(b)] with a gain proportional to the second 
derivative, the amplitude of oscillations, instead of increasing, is re¬ 
duced. This demonstrates the advantage of using such an action law. 

(2) In Figure 15(e) , a false commutation, due to the noise, effec¬ 
tively appeared at the beginning of the evolution. In spite of this fact, 
the system converges towards the optimum that illustrates the self¬ 
stability inherent to the method. 



x(t) 



h(t) 


Figure A 

Finally, we would like to point out that the study of different 
applications of this method, specially in the field of chemical pro¬ 
cesses, has led us to consider systems with a different structure as it is 
shown in Figure A. In this case the index of performance h(t) is issued 
from the product of two signals m(t) and n(t) obtained as F(p) and 
G(p), the input of two transfer functions. The study of this category 
of systems is being made at present. 
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Dual Control Theory Problems 

A. A. FELDBAUM 


Summary 

Control in closed systems, with incomplete a priori information about 
the plant, should, in the general case, include two types of action on 
the plant: (a) tentative actions, ‘experiments’, with the controlled 
plant, whose problem consists in studying it; (b) directing actions, 
whose task consists in adjusting the plant to the required operating 
conditions. The controlling action in optimum control, should, 
generally speaking, realize both of the above-mentioned tasks simul¬ 
taneously, and because of this, has a dual character. Dual control 
problems are considered in this paper. The problem is set up, and its 
general solution given, for discrete-time systems, when a change in the 
plant’s characteristics represents a discrete Markov process. Various 
examples are considered. 

Sommaire 

Le r6glage de systemes en boucle fermee, dans lesquels les informations 
a priori sur Installation a regler sont incompletes, doit comprendre, 
d’une maniere generate, deux types d’actions exercees sur cette 
installation: 

(a) des experimentations» en vue de connaitre le comportement de 
cette derniere; (b) des actions de reglage en vue d’imposer a installa¬ 
tion un comportement qui satisfasse certains criteres. 

Dans le rdglage optimal, Taction de commande doit, en prin- 
cipe, realiser ces deux types d’actions simultanement. Elle possede, 
de ce fait, un caractere dual. Ce rapport a pour objet les problemes 
de reglage a caractere dual. II pose le probleme et en donne la solu¬ 
tion gdn6rale pour les systemes discontinus dans le temps, dans lesquels 
les changements des caracteristiques de Finstallation constituent une 
chaine discontinue de Markov. Divers exemples y sont passes en revue. 

Zusammenfassung 

Die Regelung in geschlossenen Kreisen mit unvollstandiger a-priori 
Information iiber die Strecke sollte im allgemeinen Fall zwei Arten 
von Eingriffen auf die Strecke umfassen: 

a) studierende Einwirkungen auf die Regelstrecke („Experimente“) 
und 

b) steuernde Eingriffe, die die Regelstrecke in den geforderten Be- 
triebszustand versetzen sollen. 

Bei der Optimalregelung sollen im allgemeinen diese beiden 
Funktionen gleichzeitig erfullt werden, die Regelwirkung bekommt 
dadurch dualen Charakter. In diesem Beitrag werden duale Regel- 
probleme behandelt. Das Problem wird definiert und eine allgemeine 
Losung wird fur zeit-diskrete Systeme angegeben, sofern eine Anderung 
der Regelstreckenmerkmale einen diskreten Markoff-ProzeB darstellt. 
Einige Beispiele werden angefiihrt. 


Introduction 

The controlling device in an automatic system solves two 
problems that are closely interrelated, but which differ in char¬ 
acter. In the first place, on the basis of information that is fed 
into it, it clarifies the properties and state of the controlled 
plant. In the second place, on the basis of the properties dis¬ 
covered in the plant, it determines which steps have to be taken 
for successful control. The first task is that of studying the 


plant; the second task is that of adjusting the plant to the 
required operating conditions. In the simplest types of systems, 
the solution of one of these problems may be absent or have a 
primitive form. In complex cases, the controlling device should 
solve both indicated problems. Below are considered problems 
involved in the design of optimal devices that solve both problems 
simultaneously. 

Optimal systems may be divided into three types: (a) optimal 
systems having complete or the maximum information possible 
about the controlled plant; ( b ) optimal systems having in¬ 
complete information about the plant, and with an independent 
or passive accumulation of it in the control process; (c) optimal 
systems having incomplete information about the plant, and 
with an active accumulation of it in the control process. 




z 

r 


u 


X 




B 

Figure 1 


In order to clarify this classification, it is necessary to de¬ 
termine what is meant by information regarding the controlled 
plant. In Figure i, the controlled plant B is shown in the shape 
of a rectangle; in it, x is the output quantity, u is the controlling 
action and z is a non-controlled disturbance. If the plant has 
several inputs and outputs, then the quantities x, u , z must be 
considered as vectors. 

The dependence x = F(u,z ) (1) 

may assume the form of a relationship between the values of x, w, z 
at the same moment of time either in the form of a differential or 
some other type of equation. In the general case, Fis an operator. 

Information regarding the plant is gathered from the 
following elements: ( a ) information about the plant operator 
F; (b) information about the disturbance z, which acts on the 
plant; ( c ) information about the state of the plant—for example, 
about the coordinates of the point that represents the state of 
the plant in the phase space; and (d) information regarding the 
control target. 

The last-mentioned information element should indicate 
the ideal that is to be attained, and also the ‘price’ of a deviation 
from this ideal. For this reason, the control goal may be con¬ 
veniently presented in the form of a requirement for the mini¬ 
mization of some functional Q , which depends on the character 
of the x, u, z processes, and also on some externally assigned 
process x*, the reference quantity. 

Below is a statement limited by conditions of the type: 

Q(x,x*) = min (2) 
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Let Q be called the optimization criterion. Set the require¬ 
ment, for example, that the ideal process x be identical with x*, 
and that the ‘cost’ of the deviation from the ideal be expressed 
by the formula: T 

Q = c J(>c-x*) 2 df (3) 

o 

where c and T are constants. Expression (3) is an example of 
the optimization criterion. A system called optimal in which 
the minimum of the criterion Q is realized, while fulfilling 
the additional conditions that characterize the problem—for 
example, that u and x belong to some admissible domains Q ( u ) 
and Q (a), respectively. 

Complete information regarding some arbitrary dependence 
implies absolutely accurate knowledge its. For example, com¬ 
plete information about some time function, / (r), denotes that 
its values are known for any arbitrary values of t. It is considered 
below, that complete information is available regarding the 
operator F ' and also regarding the optimization criterion Q. 
All that is unknown and unforeseen in the plant is attributed to 
the disturbance z, and what is unknown in the control goal—to 
the reference quantity x*. 

Theories of optimal systems, with complete information 
about the plant, were developed in a number of papers 1 ’ 4 . In the 
theory regarding systems with an independent accumulation of 
information about the plant, consideration was given, for the 
main part, to open-loop systems. Here statistical study methods 
were introduced 1 ’ 5 > 7 . 

Problems have been considered 8 ’ 9 that pertain to the third 
type of optimal system theories. The theory involving systems 
of the third type contains characteristics that are common both 
to theories of the first as well as to those of the second type, and 
which therefore unite them to a certain extent. However, the 
third type of theory is also characterized by its own specific 
features. 

The block diagram, previously studied 8 ’ 9 , is shown in 
Figure 2. A closed-loop system is considered, in which the oper¬ 
ator of the plant B and the optimization criterion Q have been 
assigned. Plant B is acted on by a random disturbance z, which 
cannot be measured directly. The controlling action u is ad¬ 
mitted from the controlling device A to the plant B } through 
the connecting channel G , where it is mixed with the random 
noise g. For this reason, the action v, at the input of the plant 
B, is not equal, generally speaking, to the quantity u. Further, 
information regarding the plant’s state x goes through the 
connecting channel H, where it is mixed with the random 
noise h. The output y of the connecting channel H is admitted 
to the input of the controlling device A. Reference quantity x* is 
also admitted to the input of device A through channel H* 
with noise h*. 5 



Figure 2 


In the circuit shown in Figure 2, processes are possible that 
have not been considered in the theories of the first two types. A 
study of disturbance z, i.e., essentially, of the changing char¬ 
acteristics of plant B, may be carried out in the circuit of 
Figure 2, not by means of passive observation, but through an 
active method, by means of rational experiments. The plant is 
‘felt’, as it were, by the u actions, which have a perceptional 
character, and the results y of these actions are analysed by the 
A device. The purpose of these actions is to promote a more 
rapid and accurate study of the plant’s characteristics; this can 
develop a better principle for controlling it. 

However, the controlling action is necessary not only for the 
purpose of study, but also for directing and adjusting the plant 
to the required operating conditions. Consequently, in the 
circuit of Figure 2, the controlling actions should have a dual 
character; to a certain degree they should be studying actions, 
but they should also be directing actions. That is why the theory 
underlying systems of this type is called a dual-control theory. 

It is precisely this duality of control that constitutes the 
physical fact distinguishing the third type of optimal systems 
from the first two. In the first one, dual control is not necessary, 
in so far as the controlling device, without it, possesses complete 
(or maximally possible) information about the plant. In the 
second type of system, dual control is impossible, because the 
information is accumulated by means of observation alone, and 
the rate of its accumulation does not depend at all on the strategy 
of the controlling device. 


Setting Up the Problem 

Theoretical problems are considered below only for systems 
that are time discrete. All the quantities indicated in the circuit 
in Figure 2 are considered only for discrete time moments, 
^ OT, 2where n has been fixed. The value of any 
arbitrary quantity for an s discrete time has been provided 
with an index S for example, x S9 x S9 y S9 etc. The transmission 
lines of all quantities are assumed to be single-channelled, and 
the plant B is considered to have no memory. Therefore, its 
equation may be written thus: 


x s =F(v s ,z s ) ( 4 ) 

A generalization of the conclusion set forth below, for more 
complex plant cases, with several inputs and outputs, and also 
for plant having a memory, may be carried out in the same way 
as was done previously 8 . 

Let h S9 h s and g s represent a series of independent, random 
quantities with invariable distribution densities: P(h «*) P(h ) 
P(g s ). Further, let: & * ^ a> ' 


J Z s-Zs(s 9 p s ) (5) 

and / 

X*=x*(s,l s ) (6) 

where the vectors p s and l s , in contrast with refs. 8 and 9, are 
not random quantities, but discrete vector Markov random 
processes. In other words, the ju s and l s are vectors: 

and A=(^-,/C) (7) 

( 8 ) 

and fil, in the general case, are mutually interrelated, scalar 
Markov discrete processes. The same holds for and 7j. 
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However, the vectors ji s , X 89 and also, noises h *, h S9 g s are 
considered independent. 

Consider the Markov process characteristics Jl s and X s as 
having been given. This implies that one has been given, as the 
initial probability densities, P Q (/7 0 ) and P 0 (1 0 ) where t = 0, as 
well as the transient probability densities P (jk+dUd and 
P (lz+2/^)- 

The methods for combining the signal and the noise in 
blocks H* 9 H and G are considered as known and invariable, 
and the blocks themselves as having no memory. Therefore: 

v s =v s (u s , g,);y't = y't C h*,xf);y s =y s (h s ,x s ) (9) 

The control goal is determined in the following manner: let 
the specific loss function (the ‘cost’ of deviation from the ideal), 
which corresponds to the s time, have the form: 

W s =W s (s,x s ,xt) (10) 

Further, let the overall loss function W, for the entire time 
period n, be equal to the sum of the specific loss functions: 

W s (s,x s ,xT) (11) 

5 = 0 

A system is called optimal for which the average risk R 
(mathematical expectation M of the magnitude W) is minimal. 
The risk magnitude is expressed by the formula: 

R = M{W}=m\YW s (s,x„X*)} =EM{H S }= £ R s (12) 

(.5=0 J 5=0 S=0 

The expression R s = M(W s ) is called the specific risk in the s 
cycle. The quantity R plays the part, here, of the optimization 
criterion Q. 

Introduce the time vectors (0 < s < ri): 


Ug (Uq, Ux, . 

■ • J ^s) 9 

x s — \ x 0> * 



v s =(v 0 ,v u .. 

•5 V s ) , 

y s =(yo,yu- 

••=y s ) 

(13) 

X s = (x 0 ,Xx, . 


Fs —(Fo? F 19 • 

• *5 Fs ) . 



and the matrices of vector p s , vector X S9 which are made up of 
the vector columns of /Z s , X s : 

Ms “(/*()> Pi* ***5 P's) > ^S == (^05 ^s) 

Consider that the control device, in the general case, possesses 
a memory. In addition to this, assume, for general purposes, 
that the algorithm of this device is a random one. The term 
‘random strategy’ is also employed. This implies that the value 
u s is a random function of the quantities y t which were admitted 
to the input of device A during the preceding moments of 
u t (i < s), and also of the values yf (j < s). It is required to 
find the optimal probability densities: 

p s ( u $)—r s (u s \ w s _ l3 T s -i?ys)(o<s<») 0i 5 ) 

The problem consists in finding such a series of functions F s , 
in the case of which the average risk R will be minimal. Inasmuch 
as r s is the probability density, therefore: 

I r s (u s )dQ (u s ) = l (16) 

0(u s ) 


where Q (u s ) designates the region for the magnitude changes 
u s , and d Q (u s ) represents its infinitely small element. And thus 
it must be found that the optimal functions F s > 0, which are 
limited by condition (16). 

Derivation of the Basic Formula 

First write the formula for a conditional, specific risk, r 89 
understanding by this a risk in the s cycle, with a fixed ‘pre¬ 
history’ of the control device inputs, i.e., with fixed values for 

—> —> —y 

y s *, y s - 1 , u s-i : 

r s = M {W s \y*,u s -u y s -i} 

= J W s [s, x s , x* (s, J s )] • P (l s , x s | y 5 *, u s _ ! , y s - i) dO (I s , x s ) 

nO-s, Xs) ( 17 ) 

Here Q (l s , x s ) is the domain of changes for l s and x s , and 
d Q (l s , x s ) is its infinitely small element; P (I s , x s /yf ,~u g-x, ys-i) 
is the conditional, common probability density of the l s and 
x„ with fixed vectors yf, h s -i, y s -x- l n conformity with a 
well-known theorem of the theory of probabilities, an equality 
exists: 

P CL 9c s |y s ? i, y s —x) 

= P (Llks’ 1’ ks — l) ’^(^sl-Vsl Js-lJ^s) 

=P(^s\y*y p (. x s\y*^s-i>y,) ( 18 ) 

The last transformation is accurate, because the probability 
density of l s , with a fixed vector yf, will not change if vector 
"Hg-!, vector yg-x are also fixed (see Figure 2). Further, the proba¬ 
bility density of x s with a fixed vector yf, will not change, if in 
addition l a is fixed. The second multiple (18) is rewritten in an 
expanded form: 

P(x s \y?,u s -i,y s -i) 

=j p(xjft,«,) p s (K) Ps(fiJiyi 1 , y.- 1 ) Of,, u s ) (19) 

& (Ust Us) 

where Q (fi S9 u s ) is the domain of changes forj£ s and u Si and 
P s (jZ s ) is the a posteriori probability density of p s in the s cycle: 

P s (ps) == 'P(/^sl w s-l? Ts-1» Fs) 

=jp(A,lA-i)P(fi,-ily,-i.-1, yt) (fi s - 1) (20) 

i) 

Inasmuch as: 

■P(A-ilf s -i»« s -i»T s ) 

= J pjfl s - x|y s _ x, M s _ify s ) do (A- 2 ) ( 21 ) 

& (Jis - 2 ) 

where Q (/c s _ 2 ) is the domain of changes for the matrix (/£ s - 2 ), 
therefore, it is necessary to find the conditional probability 
density of P y, rt , A-i, L*)> for the matrix From the 

equality: 

P(jS s -i,u s _i,y s _x|yf) 

= P (m s _ x, y s _ x.|)i s - ij Fs)' P 

— •P((U s _i|ii s _i,y s _i,yf)’-P(u s -i) ^s-ilFs) (22) 
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one finds (because /l s -i does not depend on ~yf): 

4-1 = J*) 

_ f (“»+!» 

Here P ( Ms-!, yf) is the common a priori probability 
density of vectors u n , y^, with fixed Jf ; P 0^) is the a priori 
probability density of the matrix^, and P (Vi, y s -ilFs-i,~yf) 
is the conditional probability density of vectors ~u s - 1 ,~y n , with 
a fixed matrix p s —i and vector ~yf (the likelihood function). In 
passing (s — 1) times around the closed loop in Figure 2, it is 
possible, as in the derivation 8 - 9 , to find the expression: 


P-i (As— i) —“ 


p o (Po)UP(M^D 


~S~ 1 

Tl p (yi\FiMi) 

.i — 0 


s — 1 

n a 

_i = 0 


P( u s-i> y s -il y*) 


(24) 


The substitution of (24) in (21), and further on, of (21) in (20), 
(20) in (19) and (19) in (18) will make it possible to determine 
the second co-factor in (18). Now consider the first co-factor 
of this expression—the a posteriori probability density of l s : 


Ps(z s )~p(w:) 


Inasmuch as: 


(25) 


p (4 yT)=P (4 • p (y? 14=P (l s |y s *) • P &*) 

therefore: 


formula (24) and^(29) for the a posteriori probability densities 
of the matrices and l s . The accumulation of information 
regarding the disturbance z or the vector p s , i, e., essentially, 
regarding the unexpected manner involving the changing char¬ 
acteristics of the plant, is expressed in the fact that the a priori 
probability density P 0 (pt 0 )Js replaced in each new cycle by the 
a P os ! e ™n densities P s (pi s ) [see (20), associated with the ex¬ 
pression (23)]. From (23) and (20) it is evident that the function 
Ps (ps), and, consequently, also the rate of information accumula¬ 
tion depends on all the preceding strategies JT (i < s). In other 
words, the rate involved in studying the plant depends on how 
efficiently the experiments were set up with respect to studying 
this plant, feeding it with the actions and making analysis of 
the plant y i reactions to these actions. But in formula (25) for 
P*s which is associated with (29), a dependence of the infor¬ 
mation accumulation rate, with regard to the vector X s , from 
the strategies F i9 does not exist, i.e., the information accumula¬ 
tion process is a passive or independent one. 

By carrying out all the substitutions indicated above and 
then substituting (18) in (17), a final formula may be 
arrived at for the conditional, specific risk r s . If the values of r s , 
are considered in different experiments carried out with this 
system, then the vectors and jv-j, which, generally speak¬ 
ing, are^ not known beforehand, may assume different values. 
Let P(yf, u s - l9 js-i) be the density of the common distribution 
of these vectors; in such a case: 

u s-i>y s -x) := P(Us-i> ys-i\y*)P(y*) (30) 

In that case, the specific risk R s , which represents the average 
value of r S) where experiments have been conducted on a large 
scale, is determined by the formula: 


p s (l)= p a s \y!)=p(X s ).r$l^ (26) 

p (y*') 

The a priori probability density of the matrix X s is determined 
from the formula that is accurate for the Markov process: 

p d)= p a 0 ,4- ~,4 


= p oa 0 )- p a 1 ix 0 )-pa 2 ix 1 )...p(i s ii s _ 1 ) 

= p oao) n p m-i) 

i = 1 


(27) 


R s =M{r s } = $r s P(y*,u s _ t , y s _ J d£3 (y*, u t . u y,_ t ) (31) 
—> —> 

®(ys, u s -i,y s - 1 ) 

= J'- s i , (« s - 1 ,£-ily s *)P(;;) dQ(y* i£_ 

-> -> —> 

&(yt, u s -i,y s -i) 

Having substituted here the expression for r s , the following 
formula is arrived at: 


J ^ [ s > (s, X s ) X J • P 0 (2 0 ) • Yl P (Xi\%i -i)'P (x s \pi^ u s ) 


-> _> _> 

& (X s , fis * x s , J 7 ** Us, ys - 1 ) 


i — 1 


Further, the conditional probability density is: 

7 5 (T*l4=P(^|A 0 )-PO; s ;il 1 )... P(yf\X s ) 

= n P(yf 14 

i=0 

Consequently, from (26), (27) and (28), one obtains: 

Po(Xo)fipm- 1 y[[p(yT\xd 


Ps(l)=- 


i — 1 


i-0 


p G ,*) 


(29; 


By substituting (29) in (25), the final formula for P, (L) i 
found. 

Attention is now turned to the principal difference betweer 


i = 0 

(32) 


I"I Rj Pi ys 5 U S1 Ts~ l) 

i — 0 


It is important to note that, although in the given case* 
plant B has no memory, nevertheless risk R s , in an s cycle, 
depends on all the F t strategies at the moments / = 0,1, 

The physical reason for this phenomenon, which is absent 
in an open loop system, is found precisely in the duality of 
control. Control at a A; moment of time should be-calculated 
not only with a view towards decreasing the specific risk R lc , 
which corresponds to this moment, but also towards promoting 
a risk reduction, i?,,- (/ > k) 9 during the following moments, by 
means of a better study of the plant. 
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Determination of the Optimum Strategy 

In determining the optimum strategy 8 ’ 9 our thoughts are 
drawn towards dynamic programming 1 . Therefore introduce 
some auxiliary functions oc k (0 < k < n): 


a ~' ot k (y*,u ki Uk-i' ) y k - 1 ) 

={w k [k,xt(k,x^, 3cj-p 0 (Xo)- n pm-i)p(xM,Uk) 

j i 


£2 (X k , fi k , x k ) 

■YlPiyt^yPo^-TlPr^i-x) 


i - o 
fc-i 


k 

n 

i = 1 


(33) 


n P O',lj“i. U «i) dfi (I t , fl k , x k ) 


i = 0 

Also, let: 


(fc=0,1, ...,n) 


A-nr, 

i = 0 


(34) 


In that case, the formula for the risk i? n , which corresponds 
to the moment f = n, will assume the form: 


K»= J«»(?»*«».“»- Pn-i'Pn^(y*,u n ,y n -i) (35) 

——v>- 

n ()>„*, y„-i) 

= f Pn-lXn (y*> 1 . 1 ) d & (y»* 1 > f»- l) 

-> -> -> 

Mn-l.yn-l) 

where: 

XnGn, 

= J «nOn> u n> «» -1 > l) L, Gn, W„ - 1 , ?»- l) d & («„) (36) 

r? (Mn) 

On the basis of the theorem of average value, taking (16) 
into consideration the following can be written: 

In =(«„)uvI r„dQ («„) = (a„)av>(a„) min (37) 

n (m„) 

Assume that all 2L (z < zz) are given and that the control 
process, right down to the moment t = n, has been realized. 
The selection of F n must be in such a way as to minimize R n . 
This may be accomplished if, for any arbitrary vectors ~y* 9 % v - 19 
~y n ~ a , r n is selected in such a manner as to have function % n 
minimal. Let y n equal oc n and u n * be the value u n that minimizes oc n . 

y*==y n (0=a»(M*)=mina„(u„) (38) 

u n eQ(u n ) 


It is very simple to prove the accuracy of expression (40). 

By substituting it in formula (35), one obtains, by virtue of the 
known property of the d function: 

Xn ~ tnitl 0C n (ji n , y„ 9 U n —i, y n -i) (^Omm (41) 

Un sQ ( Un ) 

But, according to (34), this is actually the lowest possible 
value for % n . Consequently, F n * represents the optimum strategy. 

In order to find the optimum strategies where i < n, 
one must shift gradually from the terminal moment t = n to 
the beginning—see references 8 and 9. 

As a result, the rule stated below is arrived at for the deter¬ 
mination of the optimum strategy r n *~ l6 . Introduce the function: 

Jn — k ~ Tn-k (Tn-fc? U n-k> Jn-fc- l) ~~^n-k (42) 

4* J y'n-k+ 1 ( Mn~k + U Tn“fe+ 1? Tn-fc-f l) 

n (y n -fc } J^-k + i) 

The magnitude according to (38). Now find the 

value that minimizes the function y n ~ 7c , in which case: 


y*- k = min y„- k =y„- k (ut- k ) 

(43) 

Un - k eO ( Un-k ) 


Evidently, 


— M n — k (Tw — 7c? ^n — k — Is Tn-ft— l) 

(44) 

In that case, the optimum strategy is: 


r„*_,=<5(u„_ k -«:_ t ) 

(45) 


i. e., the optimum strategy is regular and consists of the selection: 
u n - h = u n *- k . From (44) it is evident that u n *~ k depends on the 
values of % and y h which had been observed by the control 
device during the preceding moments, where i < n — k, and 
also on y f * (j < zz — k). Consequently, algorithm (44) is realized 
physically. 

In the partial case, when the l s process is converted to a 
random value of 1, and Jx s to a random value of /Z, formula (33) 
for oc lc is simplified and assumes the form: 

«*=J w k lk,x*(k,X),x k -]P 0 (X) fl P(y?\%)- 

i=0 

k-1 (46) 

• Po (fl) n P (yi\fl> M i) P ( X k\fl’ U k) d0 & P’ X k) 

£ — 0 

If x jc * are given in advance, then the formula proves to be still 
simpler: 

k-l 

a* = J W k (fc, X k ) P 0 (fl) n P O',IA h «;) P(x k \fl, U k ) dQ(fl, x k ) 

£HH,x k ) i = o ( 47 ) 

These formulae had been previously brought out 8 ’ 9 . 


Evidently, u n * is the function of vectors y n *, u n - x and y n - x : 

u * = u * (y* 9 m „ - 1 , - 1 ) (39) 

In that case, the optimum strategy F n * is given by the 
expression: 

r: = S(u n ~ut) (40) 

where (5 is the unit impulse function. This denotes that F n is the 
regular strategy, and not the random one, in which case the 
optimal value u n = u n *. From (39) it is evident that the optimal 
value depends on the values previously observed by the control 
device A : u s , y s (s = 0,..., n — 1), and also on y ** (z = 0,..., ri). 


Examples 

Consider three examples that illustrate the above theory. 
Figure 3 shows a representation of the simplest system for 
which h s = 0, // is the random quantity and the equations have 
the form: 


v s = u s +g s 

yT=x:+t£ 


x s ^v s +F=u s +g s +p 


(48) 


Let the stochastic quantities y, g s and h s * have normal 
distribution rules, with 0 average values and cr^ 2 , a g 2 and a h 2 
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variances, respectively. Further, A 0 and o^are known quantities, 
in which case, 

x s *=A=const;.P(A)=— ~cxp\ I (49) 

<W 271 l 2 o\ J 

Let: 

W s = W s (s, x„ x*) = (x s - xt) 2 = (x s - A) 2 (50) 

As a result of basing the solution on the method described 
above (see reference 9) and with the application of formula (46), 
the optimal control rule is found in the following form: 

I yt iWa) 

- + 7T\r-- g T v (51) 

(s+1) W +(s+1) s+ fe) 

To explain the meaning of this formula, if interferences g s 
and h* were absent, then, for the purpose of obtaining an ideal 
value, x H = x s * = X, that would assure the magnitude fV s = 0, 
it would be necessary to establish the value of u s = x s * - y 
= X — y. In formula (51), the first two components yield an 
estimation for X on the basis of the observed y { * values. The 
final term yields an estimation for y on the basis of the observed 
differences (x { — u ( ). It is evident from Figure 3, in fact, that 
Xi — Ui = y + y f . Consequently, an averaging of the differ¬ 
ences (x.i - m) yields an estimation for the quantity y. With 
sufficiently high values for s, the final term of expression (51) 
is approximately equal to the arithmetic mean of the values 

(.Xi - 




Consider another example pertaining to the same circuit in 
Figure 3. Let h s * = 0, while y is replaced by y s and represents 
a Gaussian, discrete, Markov random process; in such case, 


p o (Mo) = -^=exp 

cr 0 n 



p (.FkW-i) = -;=exp 

yJ2n 


(j*k-Vk-i) 2 \ 

2crf J 


(52) 


The quantities g s and W s are the same as in the preceding 
example, and x s * = x* is a known constant. In that case (see 
reference 10) the optimal control rule has the form: 


k-l 

u't = x*~ £ e i _ k (x i -u i ) (53) 

i = 0 

The second component of formula (53) represents an 
estimation for the quantity ju s . The values of the weighting 


coefficients, e ifk9 which are computed from comparatively 
complex formulae that are not set forth here, possess the property: 

(0 <i<j<k) (54) 

e j,k 

The physical significance of this property in the optimum 
strategy (53) consists in that a lesser weight is imparted to 
information of older origin, inasmuch as it ‘becomes obsolete’. 
Thus, there takes place, in the control device A , not only a 
process of storing new information, but also a process of de¬ 
grading obsolete information. 



For the steady process, where k tends to infinity in 
formula (53), the e iik coefficients diminish in accordance with 
the law of geometric progression as the value v — k — i increases, 
i.e., as the previously measured difference (x { — « ?: ) is withdrawn 
from the current moment of time. It is not difficult to realize 
such an algorithm by means of the simplest circuit. 

The examples given above are degenerate, since they are 
equivalent to examples in which the value of the unknown 
parameter p is measured with a certain amount of error. The 
above theory, however, by means of a uniform method, makes 
it possible to examine even more complicated problems. Con¬ 
sider the system represented in Figure 4. The equations for this 
system have the form: 


=(M. + W , y. = x a + h a (55) 

The quantities g s and h s * = 0. Noise h s has a normal 
distribution with a zero average value and a variance c h 2 . 
The quantity x s * is absent (for example, it may be assumed 
that = 0). All the values of W s = 0, with the exception of 
the last one: 


W=x„ 


(56) 


Assume that ^ is a random quantity with a probability 
density of P 0 (^), in which case \jbc\ <1. In the same manner, 
it may also be assumed that | u | < 1. The problem consists in 
determining the optimum algorithm for the control device A 
that would satisfy the condition: 


R=M{W n } = M {x M } = min (57) 

The solution of this problem produces the optimal method 
for searching a minimum of the parabolic function x = (it -f ju ) 2 , 
where p is unknown, and x is measured with an h error. At first, 
where / = 0, 1,..., n — 1, probe values for u t are established, 
and the corresponding quantities are measured. Following 
this, where i ~ «, such a u n is established as to give a minimal 
mathematical expectation to the x n value that corresponds to it. 
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For the given problem: 


P (y t \n, i, u,) = —^=exp j - A? [> ; -- uf -.2 up - /i 2 ] 2 | 


: exp {a t 4- biH + c t fi 2 4- 4- //} 


where: 


_ 2(^1 ) 5 bi 2 Ui(ui yd 2 

2<J h Gh 


c t = —2(3 uf-yd; d t =- 


All ot k = 0, with the exception of oc n , for which, in conformity 
with (47) it is found that (integration for x n is replaced by the 
substitution x n = ( u n + /^) 2 ): 


a r = K + M) 


(<W 271 )" 


ex P|E (ai + biH+c^ +d i fi 3 + n )| dfi 


For this reason, 

r* = a,T= min «„ = exp {4 _ x } ©„* (£ n _ 1 , F„ _ t ) 

Un e Q ( Un ) 

where <9 n * is the function of two variables. 

Further, 

00 

y„-i = J y*dy„_ 1 =exp {y4„_ 2 }-©„(£„_ 2 , F„_ 2 ,h„-i) 

— 00 

and, if one assumes that: 

©*_!= min ©„_ x 

Un — 1 e^(u„-i) 

V„*- ! = exp K_ 2 } • ©„*_ t (E n -2, F n _ 2 ) 

(k = 0,l,...,n) 

In a similar way, one obtains (/c = 0,1, n): 

y *_ * = exp { A„ _ _ i} • ©*_ k (E„ -F„ _ fc ) 

where 

fc (J^n—k* -k) == tnin 0 n _ k ( < E n -j c , F n —j i ,U n ^.]d 

U n -j c 6 & (u n -k) 


(64) 

(65) 

( 66 ) 

(67) 

( 68 ) 
(69) 


By making use of this expression, it is possible to find y £ and 
the values for u £ *, which, in minimizing^, prove to be optimal. 
This is accomplished by means of a succession of alternating 
minimizations and integrations, in the course of which, it is 
necessary to memorize the functions of three variables that are 
called, as is known (see, for example reference 11) sufficient 
coordinates, but the functions of three variables are too complex. 
For this reason, in the given case, the sufficient coordinates 
prove to be, figuratively speaking, insufficient for a convenient 
solution. However, the solution may be considerably simplified. 
As was indicated by calculations, by means of expansion into 
a Pike and Silverberg series 12 , it is possible to assume, with a 
sufficient degree of accuracy: 

+ c ^ 2 + difj 2 + /x 4 = 

= (pi (yn «i) A (aO+ <P2 (y-n «,•) h 00 ( 61 ) 

where <p l9 f l9 cp 2 , / 2 are some functions. In that case, 
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where 

s 

A s = X «i=A-i + a s 

i-0 
s 

E s = X <Pi(yi’ u i)= E s-i +( Pi(ys’ u s) l (63) 

i — 0 
s 

e s = X 9 ) 2(yi ! Mi)=F s _ 1 +<p 2 (y s >M s ) 

f = 0 


And so it is seen that it is only necessary to memorize the 

7 c functions of two variables, which can be accomplished 
without any considerable difficulties. In minimization, it is 
sufficient to verify the extreme values of u n - k = ± 1. 

Conclusion 

The dual control theory may be extended in various direc¬ 
tions. Thus, for example, its extension to purely discrete systems, 
in which each of the quantities can assume only one of the 
permissible levels merits attention. 

The development of this theory makes it possible to clarify 
the principles involved in the optimal teaching of discrete 
automatic machines. 

The theory described above pertains to the ‘Bayes’ type, 
inasmuch as the assumption is made in it, that the a priori 
probability characteristics are known. However, the formulation 
of the dual control theory is likewise expedient for those cases 
where these characteristics are unknown. Such a formulation 
may be carried out either on the basis of the minimax principle 
or by the application of the idea of inductive probability. 

At the present time, the most important problem for the 
immediate future is the development of approximate solution 
methods for dual control theory problems, the formulation of 
sub-optimal strategies, the determination of the numerical value 
of risk in quasi-optimal systems and its comparison with 
the value of risk in existing systems. Such a comparison will 
make it possible to clarify the extent of the gain that may be 
anticipated where we have a maximum degree of perfection 
in existing systems. 
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DISCUSSION 


L, F. Kazda, Electrical Engineering Dept., University of Michigan 

Ann Arbor , Michigan , U.S.A. 

Professor Feldbaum is to be complimented on his fine paper, which 
presents an extension of the author’s companion papers published in 
Automat. Telemech. in 1960.1 would like to ask the following questions: 

(1) Plave any practical systems been analysed, utilizing the ideas 
presented in this paper, to evaluate the average risk in a specific 
operating situation ? 

(2) Have you given any thought to the case when the plant contains 
time delays or memory? 

(3) Since for any reasonable size of plant one generally is limited 
by computer capacity, have you considered sub-optimal strategies or 
approximations to a physical plant to give a usable solution? 

A. A. Feldbaum, in reply 

I am grateful to Professor Kazda for the interesting questions he 
presented in his discussion. 

(1) We have not yet considered any practical systems but we have 
solved several theoretical problems which in principle are of con¬ 
siderable interest. The results proved to be simple and give us hope 
that in practice also the solution of these problems will not be too 
complicated. 

(2) One of my colleagues developed the dual control theory as 
applied to plants containing time delay. This paper will appear in the 
near future in Automat. Telemech. As for the plants containing memory, 
i.e. dynamic plants, they were investigated in some of my works. 

(3) For several simple cases we have constructed optimal strategies, 
which proved to be extremely simple. They were obtained by the 
approximation of curves found on a digital computer for an exact 
optimal strategy. 


S. Paszkovski, Institute of Automation , Polish Academy of Sciences , 
Warsaw, Poland 

Dual control theory presented in this paper is of great importance in 
controlling the plant with incomplete information on the plant’s 
parameters or medium. I understand that the whole dual control 
problem can be subdivided into the three following problems: 

(1) The search of algorithm when the information is known (the 
search of decision strategy). 


(2) Determination of methods for accumulating the information in 
unknown parameters. 

(3) Determination of methods for selecting the control actions 
(taking into account losses and accumulation of information) in order 
to optimally approach the decision strategy. 

In practical examples I obtained the following: the parts of the 
system which simulate (with accumulation) unknown parameters 
made it easier to realize dual control. 

A general schematic of the system is shown in Figure A. 

Let us assume that the automation operates according to the 
optimal algorithm in the case where complete information is supplied 
to the input. However, we do not have the complete information and, 
therefore, it is necessary to study unknown parameters of the medium 
and the plant. It is necessary to control the system in such a manner 
that the decision strategy could be obtained with minimal additional 
losses. 

In actual examples, the plant has a finite number of states and the 
transient probabilities are the unknown parameters. 

P{x s +i/x s , u s j 

An estimate of unknown parameters has been carried out over 
intervals. Such a method made it possible to realize dual control which 
proved to be close to the optimal. 


A. A. Feldbaum, in reply 

A technically sound way to solve the problem of constructing a con¬ 
troller often consists, especially in complex cases, in dividing the 
overall control task into two problems ; 

(a) Determination of the unknown parameters of the plant, which 
is performed by separate units, e.g., with the aid of an analogue. 

(b) Determination of the control action from the parameters found. 

However, in the general case this method is not optimal. In the 
optimal solution of the problem the control has a dual nature and both 
the above problems are solved together. In the general case it is not 
possible to divide the optimal controller into two separate units of the 
kind mentioned above. It is, however, quite possible that such a 
division may, in many cases, make it possible to implement a strategy 
close to optimal; in some cases a strictly optimal strategy may even 
be possible. 



Figure A 
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DUAL CONTROL THEORY PROBLEMS 


H. Chestnut, General Electric Company , 1 River Road, Schenectady, 
N.Y., U.S.A. 

To what extent does Professor Feldbaum find, for problems of prac¬ 
tical systems, that analytical methods of solution are applicable, and 
to what extent are machine methods (computers) more satisfactory ? 
Although not specifically referred to by Professor Feldbaum in this 
paper, there is the question to what extent the system designer per¬ 
mits the noise to be a significant factor and having to be taken into 
account, and to what extent is it worth while for him to try to reduce 
the noise to an acceptable amount. 

A. A. Feldbaum, in reply 

In his comments Professor Chestnut mentioned two extremely inter¬ 
esting questions, on which the following can be stated: 

(1) The computers can be used in various ways: 

(a) Obtaining the results of solution of a theoretical problem, for which 
a general solution specification is given; for example, in dual con¬ 
trol theory problems of a non-trivial nature can be solved only 
with the aid of computers. 

(b) Obtaining results by search in those cases when there are no analyt¬ 
ical methods of solving the problem. For this one can use either 
universal digital computers or other equipment—for example, the 
automatic synthesis equipment designed at the Institute of Auto¬ 
mation and Telemechanics, Moscow. 

(c) An optimal or sub-optimal algorithm, however found, can be 
put into a computer—digital, analogue, or hybrid—which in the 
given case will be the controller. 

The proportion of purely analytical methods of problem solving is 
now comparatively small, and will evidently grow smaller. In the near 
future solving a problem will come to mean finding the programme by 
means of which the problem can be effectively solved on a computer. 

(2) If the controlling plant itself, and also the meters and correcting 
devices, are specified—as is most often the case—the system designer is 
forced to take into account random interference and noise in the com¬ 
munications channels, as certain quantities with specified properties, 
which he cannot reduce at will. 

Of course, he can, for example, put a filter in the feedback circuit 
leading from the plant output; however, filtering also causes an un¬ 
desirable lag effect, and it is not clear in advance whether inclusion of a 
filter will lead to improvement or impairment of system behaviour. 

Optimal choice of such a filter is also part of the synthesis of an 
optimal controller. 

It is impossible to give general rules about when noise is significant 
and when it can be ignored. I can only give an example when even a 
small noise can have considerable influence. Optimal processes in a 
high-order system are only possible when the controller knows the state 
of the system, i. e. the values of the higher derivatives of its output. 
But how can they be obtained ? 

In many cases it is not possible to extract them from the controlled 
plant. All that can be done is to differentiate its output. However, in 
differentiation even the smallest noise mixed with the plant out¬ 
put signal leads to great noise in the values of the higher derivatives. 

Therefore, it may turn out that failure to take noise into account 
in these cases will lead to a considerable deviation of processes from 
optimal. 

A. M. Hopkin, University of California, Berkeley 4, California, U.S.A. 

Since application of this dual control calls for a rather elaborate con¬ 
troller, the practical problem of cost of the controller as compared to 
the advantage gained must be considered. I would like to ask Pro¬ 
fessor Feldbaum to what type of progresses he foresees applying this 
technique, and what applications have already been made. 


A. A. Feldbaum, in reply 

There are as yet no practical applications of the theory of dual control 
for the creation of real controllers. 

The question of their cost is therefore still very much a matter of 
speculation. However, two things inspire hope that the cost will not 
be excessive: 

(a) For some calculated cases sub-optimal strategies proved com¬ 
paratively simple and realizable by means of uncomplicated circuitry. 

(b) If the algorithm is realized by means of programme-control 
computers, making the algorithm more complex will only involve in¬ 
creasing computing time, not making the controller more costly. 
Therefore, if the computing time is acceptable, the cost of the con¬ 
troller does not depend on the complexity of the algorithm (until the 
complexity and the consequent slowdown of computations pass per¬ 
missible limits, requiring a faster and more expensive computer). 

B. Friedland, Aerospace Research Center , 1225 McBride Avenue, 
Little Falls, U.S.A. 

(1) The distinction between the last two types (b) and (c) of optimal 
systems is not clear to me, particularly in view of the second para¬ 
graph in the second column of page 541. 

(2) It appears that the extension of the approach of statistical 
decision theory represented by this paper to dynamic plants subsumes 
many of the problems now being considered in the theory of stochastic 
optimum control. Would the author care to discuss the following: 
(a) Model used for dynamic processes; (b) Comparison of his results 
with those of other investigations in the U.S.S.R., U.K., and the 
U.S.A.? 

A. A. Feldbaum, in reply 

(1) The difference between systems of types B and C is the same 
as that between observation and experiment. In B type systems it is 
only possible to observe the output of the source of random signals; 
it is not possible to act upon it. In C type systems, study of a plant 
with random properties can be forced by feeding to it control actions 
such that analysis of the reaction of the plant to these actions enables 
it to be studied faster. 

Here the controller in essence performs ‘experiments’ with the 
plant, which cannot be done in an open-loop system. 

(2) Extension of the theory of dual control to dynamic plants was 
made in another of my papers 1 . Comparison with the results of other 
approaches has been made only for certain very simple, so-called 
‘reducible’ systems, i.e., systems for which an equivalent open-loop 
circuit can be built. The comparison showed the results to be identical. 

As far as basically closed-loop systems are concerned, it is not 
possible to draw a comparison since such systems have not so far been 
studied. 

Only one example has recently been examined 2 . 

Comparison of the results of dual control theory with the results 
of studies in which two operations—determination of the parameters 
of the controlled plant and finding the control action from the para¬ 
meter values obtained—are performed by separate units, is of 
undoubted interest. 
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D. Xirokostas, Electrical Engineering Department, University of 
Birmingham, Birmingham, England 

First, I would like to say that I found the paper very interesting and 
important because it gives a strict mathematical formulation and 
solution of the general discrete optimal control problem when non- 
linearities, dynamics and random disturbances or noise are simul- 
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aneously present in the process; furthermore, because the presented 
theory, m my opinion, helps in the solution of some particular 
problems of this kind, it is of great practical importance. However, this 
is not indicated in the paper and I think more work in this direction 
is needed to clarify the field of application and the practical limita¬ 
tions of the theory. 

. The first P° int of m y discussion refers to the last example, shown 
in Figure 4 of the paper, in which a non-linear process is considered, 
say of a simple parabolic form, and in which random disturbances p 
m the input and noise f s in the output are simultaneously present. 
In the system, the optimum controller A derives informations about 
the process output x s and uses it to produce the optimum control law 
m terms of the assumed performance criterion R . Although both 
random signals p and £ s have been assumed stationary, the effect of p 
sensed by the controller at the point H, because of the intermediate 
non-linearity, is a non-stationary signal, depending on the value of 
the control level u s . Furthermore, this signal is not any more normal 
although this difficulty is not very important. Because of the first 
difficulty, however, the controller has to deal with a non-stationary 
signal depending on u s which I think makes it difficult to derive the 
optimum control law for it. It is not clear to me how this has been 
taken into account in the paper, and I would like the author’s comments 
on this point and ask if he has some more information about it, or has 
dealt with any particular example. 

My second point is to ask if, in this particular example, process 
dynamics have been considered. In this case the problem becomes 
much more difficult, because the optimal decision at each step depends 
not only in the present state of the system, but also on previous ones 
and so the Markoff process describing it is of higher order. 

The solution of this last problem has been tried by the speaker, 
for the case that p disturbances are not present or they are simple 
step or ramp functions in an otherwise realistic model. The employed 
criterion is the same as the risk function R used in the paper. There 
have been two methods tried for this problem. One method is analyti¬ 
cal, using the Markoff process theory. Because, in the problem, the 
Markoff process is of higher order, it is turned to that of first order 
by introducing new variables and so extending the transition matrix 
of the process. Then the conventional theory of Markoff processes 
is applied. 

The other method is a ‘Monte Carlo’ method. The whole system 
is simulated in a digital computer, making use of the facility of produc¬ 
ing random numbers inside the computer of a desired distribution 
and variance. It is hoped that results from both methods will be 
published soon. 

Thinking in terms of including the random disturbances p into 
the model, it seems that the above approach can be extended to the 
case where p consist of a Markoff process of first order and this will 
be the next step of the research. 

It must be pointed out that the solution pursued above is sub- 
optimal because a stepping strategy with constant step in u s is assumed 
However, this assumption can be relaxed, keeping always the discrete 
nature of the controller, and so pursue optimal solution. Although 
this optimal solution is very interesting from a theoretical point of 
view and from the point of view of comparing its results with those 
of the previous solution, it does not seem that, at present, it can be 
easily applied to practical systems. 


A. A. Feldbaum, in reply 

(1) Different idealizations of the system and formulations of the 
problem are possible. For example, in many problems of statistical 
theory, stationary random processes are examined. In the paper in 
question, however, another approach is used, and a stationary state 
is not required. In Example 3 the parameter p is a random quantity. 
The general formulae given in the paper enable one to find the solution 
specification if p is a Markov random process, not necessarily 
stationary. 


(2) I fully agree that to take the dynamics of the plant into account 
makes the solution process complicated. In principle, however, the 
method of solution remains the same. A general solution specification, 
taking plant dynamics into account, was given by the author in 
Automat. Telemech ., No. 1, 1961. I would be very interested to learn 
what Mr. Xirokostas has done to solve a similar problem. 


O. L. R. Jacobs, Department of Electrical Engineering , University of 
Edinburgh , Edinburgh , Scotland 

Professor Feldbaum has presented a very general method of formu¬ 
lating problems where it is necessary to learn about a process as it 
evolves in time. A rather academic, but classical example of this 
type of problem, is that known as the two-armed bandit. 

I would like to ask Professor Feldbaum whether this problem has 
been formulated in the language of dual control theory and whether 
it has been possible to derive solutions to it. 


A. A. Feldbaum, in reply 

Dr. Jacobs pointed out one interesting possible development of the 
theory of dual control. Bellman gave the principle of the solution 
of the two-armed bandit problem, using dynamic programming. 
An important feature of this problem is that the magnitude of the 
probability characterizing the plant is unknown, and only an a priori 
distribution is given for it. 

For some classes of systems the theory of dual control has already 
been extended to similar problems, in which the a priori distribution 
of the unknown parameters of the plant contains unknown coefficients. 
In its turn the a priori distribution for these coefficients is specified. 

The solution of the problem makes it possible to build an optimal 
adaptive or self-learning system, which accumulates information 
about these coefficients and clarifies their values. 


P. Dorato, Polytechnic Institute of Brooklyn , 333 Jay Street, Brooklyn 
1, New York, U.S.A. 

I should like to ask Professor Feldbaum the following questions: 

(1) Has any study been made of the case n — oo , in particular has 
any study been made of the controllability of a system (in the sense that 

exists) ? 

(2) What ingredients must go into a control problem to make the 
strategy stochastic? Has an example of a stochastic strategy been 
worked out? If the strategy is stochastic how can it be implemented 
in a practical way ? 


A. A. Feldbaum, in reply 

(I) In the problem of optimal control of a Markov plant (Example 2 
in the paper; see also Automat . Telemech., No. 8, 1962) an examina¬ 
tion was also made of a steady-state process, in which the mean risk 
per cycle is reduced in the same sense as in the question. 

(2) Generally speaking the optimal strategy will be random, if the 
optimum criterion is selected not in the form of the assembly average 
of the loss function, but in the form of its maximal value in the ‘worst’ 
case (a problem of the ‘minimax’ type). Nothing has so far been 
published in this direction. An optimal controller with random 
strategy can be implemented with the aid of a random signal generator 
and certain extra units. These devices are realized in the form of not 
very complex circuits. 


lim 

n-+ oo 
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The Application of Random Test Signals in Process Optimization 

P. M. E. M. van der GRINTEN 


Summary 

In the determination and continuous adaptation of the optimum set¬ 
tings of a continuous process the influence of disturbances and dynamic 
effects can be ruled out by using random test signals superposed on 
the adjustable parameters. 

In some cases use may be made of signals that are inherent in the 
process; in others it is necessary to superpose specially generated 
signals. The two methods are compared as regards possibilities of ap¬ 
plication, need of apparatus, and results. 

Sommaire 

Dans la determination et Fadaptation continue des regimes optimaux 
d’un processus continu l’influence des pertubations et des effets dyna- 
miques peut etre eliminee par l’utilisation de signaux d’essai alea- 
toires superposes aux parametres reglables. 

Dans certains cas, on peut utiliser des signaux inherents au proces¬ 
sus ; dans d’autres il est necessaire de superposer des signaux speciale- 
ment engendres. Les deux methodes sont comparees sous l’angle de leurs 
possibility ^application, de l’appareillage necessaire et des resultats. 

Zusammenfassung 

Bei der Bestimmung der selbsttatigen kontinuierlichen Einstellung 
eines Optimums bei einem kontinuierlich arbeitenden Prozefl, lassen 
sich der EinfluG von Storungen und die dynamischen Auswirkungen 
ausgleichen, wenn man zufallsbestimmte Testsignale den einstellbaren 
GroBen iiberlagert. 

In einigen Fallen kann man hierfur die in dem Prozefi auftretenden 
Signale verwenden; in anderen Fallen ist es notwendig, eigens erzeugte 
Signale zu iiberlagern. Die Anwendungsmoglichkeiten, der technische 
Aufwand und die Ergebnisse der beiden Verfahren werden verglichen. 


Introduction 

For the experimental maximization of a function 

y=f(,x u ...,x n ) (1) 

it is advantageous to employ gradient methods, taking account 
of possible constraints. To be able to use such gradient methods 
the partial derivatives of y to Xj ... x n should be known. If y be 
the profit function of a continuous process in which x 1 ... x n 
are the adjustable variables, these partial derivatives may be 
considered to represent the static gains of the process paths. For 
the experimental determination of these gains use may be made 
of various methods, all of which are based on the same prin¬ 
ciple, viz., making small changes in the x values and measur¬ 
ing the corresponding reactions of the y signal. 

The main disadvantages experienced are (a) the mutual 
disturbance of the process paths and the influence of the inherent 
noise, and (b) the inclusion in the process of unknown and in 
some cases variable dynamic effects, which mask the pertinent 
static relation. 

A solution to these difficulties may be found m the use of 
independent stochastic test signals for x -±... x n , which makes 


possible the separate determination of the influence of each of 
the x variables on y separately by correlation. The dynamic 
effects can then be eliminated by using correlation functions 1 
and not only correlation coefficients. These correlation functions 
do not contain any information about the phase of the signals, 
so that dead time and other non-minimum phase properties of 
the process do not complicate this measuring method, in con¬ 
trast to methods using periodical test signals. 

An elimination of dynamic effects as meant here is not 
pointless, because the profit is often a complicated function of 
a number of input and output variables, on account of which 
several parallel paths can develop in the process (= plant + 
profit calculation). This may give rise to non-minimum phase 
properties that are highly sensitive to small variations in process 
or settings. If no compensation is made for this phenomenon, 
it is no longer the pure gradient that is measured, so that the 
optimization may show a hunting effect. Consequently, complete 
elimination is particularly desirable whenever a moving opti¬ 
mum is to be followed. 


Random Test Signals 


Starting from the existing relations between the auto-correla¬ 
tion and cross-correlation functions of various input and output 
variations, a general formula describing the static gain of the 
process paths was developed. For small variations the process 
is linear and correlation functions can indeed describe the 
dynamic properties of the process paths independently by means 
of convolution integrals. The relative deconvolution problem 
can be obviated if it is necessary to measure not the full dynamic 
properties but only the static gain, which can be found by 
extrapolation. This extrapolation is effected by integration of the 
correlation functions, as shown later. 

A special case is given by 


Ay_ 

Ax n 






( 2 ) 


As cj) XnXn (r) and (j) XnV (r) can be determined continuously 
it is possible using this equation to measure the static gain of all 
process paths simultaneously. The expression has been success¬ 
fully used for the continuous calculation of the gradient of the 
profit function of a gas burner, and a consecutive optimization . 
The x signals were provided by the inherent spontaneous 
disturbances. In many cases, however, it will be necessary to use 
specially generated stochastic test signals, for the reasons given 

below. _ . , 

(a) The spontaneous input variations may be interrelated, 
so that eqn (2) does not apply and the complete formula given 
at the end of the paper will have to be used. 
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(b) The spontaneous input variations may have an unsuitable 
power spectrum, which is shown by eqn (2) becoming indeter¬ 
minate. 

(c) The possibility of measuring the x signals may be lacking. 

(d) Because of their wide amplitude and frequency ranges, 
the signals obtained from the spontaneous disturbances do not, 
generally, lend themselves well to being delayed for a sufficient 
length of time to enable the correlation functions to be calculated. 

For these reasons a noise generator was designed, which 
had to generate a number of uncorrelated noise voltages having 
a stationary character and an adjustable power range. The output 
voltages are of a binary shape, so that they can be easily super¬ 
posed on the setting parameters and can be delayed by means 
of punched-tape apparatus to enable the correlation to be 
determined. Some technical and theoretical details of this noise 
generator are given in the Appendix. 

Optimizers 

The principal element of an optimizer operating according 
to the ideas outlined above will have to be a computer for 
eqn (2), i.e. for determining the integrals of correlation functions. 



Figure 1. The continuous measurement of the integral of correlation 

functions 


Consider the case in which use can be made of the inherent 
noise. The usual delay line for the calculation of correlation 
functions can be avoided by making use of a circuit as in Figure 1. 

In this circuit the ^ signal is filtered by a number of RC 
filters with different time constants; special weights are applied 
to the output voltages which are then multiplied by y. 

It can be demonstrated (see Reference 1 or the more simpli¬ 
fied reasoning shown in the Appendix) that the result is actually 
an approximation of the integral of <j b xy (t) for 0 < r < T. 
The number of RC filters is determined by the accuracy re¬ 
quired and by the magnitude of T. 

Physically interpreted, the operation of the ‘static filter’ S 
consists in a determination of the moving average of the x signal. 
This may be expressed as 

* ( o4j>-, )d , (3) 

The result, p, can now be calculated in a simple way: 


P=E t b0)^(0]=Y 


E c \x(t— x) .y (0] dx 


4j: 




Whether this expression, as it stands, can be used in eqn (2) 
will depend on the character of the correlation function before 
x = 0 and after r = T. If before r = 0 there are already signifi¬ 
cant values, these must be calculated separately with the help of 
an arrangement according to Figure 7, in which x and y have 
been interchanged. The ‘averaging time 5 Fmust not be too short, 
as otherwise an appreciable part of the correlation function 
would be outside the area, so that the result would again be 
influenced by the dynamic properties. On the other hand, 
T should not be too long, as the signal to noise ratio decreases 
rapidly at higher T values, resulting in large measuring errors. 
It can be shown mathematically that eqn (2) becomes more 
nearly indeterminate as the T value increases. 

Another method of performing the operation of eqn (3) on 
the stochastic input signal x, is the application of a sampler with 
period T } and a zero-order hold circuit. This can be shown by 
an analysis in the frequency domain. This suggestion makes it 
possible to determine the integrals of correlation functions, and 
hence the gradients of the response surfaces, by a simple manual 
calculation. The method described in the next section can be 
regarded as a mechanical and more efficient application of 
several of such samplers in parallel operation. 

On the basis of the concept of the ‘moving average’ the case 
in which special stochastic signals are added is treated. With 
a circuit as shown in Figure 2 it is now possible to fix the limits 
of the moving average also before r = 0, by superposing a 
binary signal read from the pre-punched tape. 

According to the diagram, the signal X(t) is given by 


X(t) = 


1 


oo 

x(r — t)dt — 
-r i 


x(r~t) dt= 

J t 2 ] 


% t 2 

x(x — t)dt 

-Ti 


(5) 


Correlation of this signal with y(t) again leads, according to 
eqn (4), to the desired integral. 

Arrangements as shown in Figures 1 and 2 have been used in 
automatic optimizers: the first, for a process having strongly 
varying dynamic properties, with transients of the order of 
1 min but without any dead time (a gas burner system); the 
second, for a process having transients of about 10 min and a 
dead time of 20 min, which rendered the RC filter method 
technically unsuitable. In both cases two input variables were 
optimized simultaneously by using the quantities calculated with 
the apparatus shown in Figures 1 and 2 respectively, for a con¬ 
trol of these variables. Two methods for closing the optimizing 
loop were investigated. The first consists in a directly proportion¬ 
al control of the valves of the input variables by the output of 
pure integrators which follow upon the multipliers in Figures 1 



Figure 2. Use of a punched-tape apparatus for the determination of the 
integral of correlation functions 


(4) 
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and 2. In the second method RC filters were applied instead of 
the pure integrators and the outputs were, after amplification, 
fed to limited-speed valve motors. In both methods the auto¬ 
correlation function of x(t) was assumed to be stable and was 
not taken into account. The first method proved to be slow but 
accurate, and is to be preferred for real optimizing problems. 
The second method is less accurate owing to inexact following 


of the gradient, but fast and, hence, more useful in systems 
intended for the adaptive following of a moving optimum. 

The results of some experiments performed with these opti¬ 
mizers according to the first method of closing the optimizing 
loop are shown in Figures 3 to 6. Figures 3 and 5 show a number 
of optimizing runs in a field of (estimated) lines of equal profit 
Figures 4 and 6 show the relative time diagrams. The rate of 



Figure 3. The optimization of a fast process by means of the method 
of Figure 1 




Figure 5. The optimization of a slow process by the method of Figure 2 



convergence is limited not only by stability requirements, e.g. 
the loop gain, but also by the accuracy of the measurement 
which is determined by the integration time. It can be seen here 
that with approximately equal loop gain of the optimizing cir¬ 
cuits about the same convergence rates are obtained with respect 
to the transients of the processes, irrespective of the dead time 
and the exact character of the dynamic properties. 


Appendix 


The General Formula for the Static Gain of Process Paths 

The relation between a random test signal on the nth ad¬ 
justable input parameter and the y signal is given by the cor¬ 
relation function 

<0e„j>( T ) = E 0,(0 Oc„x m (t-t)dt (6) 

m = 1 J 0 

The test signal is here assumed to be so small that the re¬ 
actions of the process are linear. 

If both members are integrated with respect to r 


f: 


3 m — 1 


0,(0 


4„x m 0-0dtdf (7) 


As h n (t) is the response to a small pulse, the partial static 
gain (i.e. the final value of the response to a small step) can be 
calculated from eqn (7) by means of 



K( t) dx = 


N 


E 


<0c„yC0dT (8) 


J -CO 


in which the coefficiency matrix oc m l is found as the inverse of 
the matrix 




”* + 00 



(0 dT 


__ 

— oo _ 


(9) 
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In eqn (2) it was assumed that the input signals x are ortho¬ 
gonal, so that oc mn is the unit matrix. Rudd 2 independently 
developed an analogous train of thought with respect to the 
frequency domain. 



Eqn (11) is identical with eqn (4) if S(r) = 1 jT in the range 
0 < r < T and S(r) = 0 outside this range. 

This shape of S(r) should now be approximated by a sophis¬ 
ticated choice of G m in eqn (12). If eqn (12) is multiplied by 
l/R m C m exp -rjR m C m and integrated with respect to r 

00 e \R n c n R m cJ 

o i?„E m C„C m dT 

(13) 

If the values of R n C n are fixed, G m can be calculated from 
eqn (13). If, for example, N = 5 and R n C n = 2~ } \ the values of 
G 1 ...G 5 are 1-00, - 1*34, 0*86, -0*34, 0*06. 




Figure 7. A generator for four statistically independent noise signals 

The Design of Statistically Independent Noise Generators 

The basis of the generator is a G.M. detector which, under 
the influence of a radioactive source, delivers pulses having a 
Poisson distribution (average v p.s.). These pulses pass through 
a number of parallel gates, which open / times per second in 
succession, for a short time T. The pulses passing through the 
gates also have a Poisson distribution*, but their average number 
is much lower. The issuing pulses every time trip a bistable 
element. The correlation function of the block voltage thus 
obtained is rendered by 

(j> xx (r)^~ 2vfT ' r] ( 10 ) 

The cross-correlation of signals from the channels is nil, as 
the pulses are completely independent of each other at their 
entry. Figure 8 gives measuring results for <fi xixi and 4> XiXy 



Figure 8. Correlation functions of the test signals generated by the 
method of Figure 7 


Nomenclature 


Evaluation of the Coefficients G n in Figure I 
From Figure 1 it follows : 

p = £ r [3;(0*X(0]=£ f [T(0- f°°S(T)*x(t-T)dT] (11) 

J o 


Here ^(t) denotes the pulse response of the filter S in Figurel : 


S(?)= £ <?„ 


L-j nor' 
n~ 1 


* In actual fact this is not quite true: the pulses are always spaced 
by a multiple of T, but T<^ 1/v. 


y Profit function 
x Controllable variable 
<Pxy (j) Correlation function of x and y 
r, T, t Time (sec) 

h ( t ) Pulse response of a linear process 
v, f Frequency (c/sec) 
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DISCUSSION 


R. Perret, 44 Avenue Felix Viallet , Grenoble , France 

May I ask the author to give further details about the last paragraph 
of his introduction: (a) About parallel paths that can develop in the 
process; ( b) About the non-minimum phase properties that appear 
in these conditions. 

P. M. E. M. van der Grinten, in reply 

By ‘parallel paths in the profit calculation’ I mean to say that profit is 
a function of both input variables and output variables. In such a 
case, a change in an input variable affects the profit along two 
different paths: directly and via the process. Take, for example, the 
gas burner system shown in Figure A. 


Gas flow G 

-1 

Burner 

Temp. diff. A T| 

• 

Profit ^ 

X (t) 

1 

system j 


F~~*i 

• j 

j 

Y (t) 


Figure A 


The response of the profit to a stepwise increase of the gas flow 
is fast in relation to the G, but is slower in relation to the resulting 
temperature change AT. There are now two possibilities [Figure B(i) 
and B(ii)]. 

In Figure B(i) there is a positive static gain, whereas in Figure B(ii) 
the gain is negative; apparently the two cases are on different sides 
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of the optimum. The responses shown are typical of such non-minimum 
phase systems. The change of the apparent dynamics as a result of the 
settings of the input variables is also obvious from this example. 

B. P. Th. Veltman, Technological University of Delft , Lorentzweg 1 , 
Delft , Netherlands 

Dr. van der Grinten apparently belongs to the most distinguished 
class of control mathematicians who have actually seen a process. 
The application of his method is very practical indeed—by asking 
the right question he eliminated the need for a digital computer 
making the methods also useful for small-scale plants. 

My question concerns the type of binary test signals that are used. 
They are statistically independent. Orthogonal signals of the binary 
type could also be used; non-linearity in the process may, however, 
deteriorate the orthogonality. Is it possible to derive some conclusions 
about the non-linearity of the process from experiments with statisti¬ 
cally independent and with orthogonal signals or are the relations 
too involved to be of any use? Could this comparison between both 
outcomes otherwise be recommended as a sensible means for establish¬ 
ing the linearity of the process? 

P. M. E. M. van der Grinten, in reply 

I agree with your statement that slight non-linearities in the process 
will nullify the advantages offered by the use of orthogonal test 
signals. Therefore, I have stated that complete independence of the 
test signals is desirable. The use of orthogonal test signals for analysing 
the non-linearities is theoretically possible, but this would require 
knowledge of the dynamics. 

H. A. Barker, The University , Glasgow W. 2 , Scotland 

The author has presented a method for obtaining a linear expansion 
of a process output in terms of the process inputs. Has he considered 


processes for which only a poor representation can be obtained in 
this way? 

There is an alternative method for obtaining these coefficients for 
a linear expansion which is an optimum representation in the mean 
square error sense. If y is to be represented by the linear expansion 

n 

£ a i x i 
i = I 

then the error e is given by 

n 

e=y~Y a i x i 

1=1 

and is such that e 2 is minimum when the representation is optimum. 
Hence the coefficients a t must be such that every average product 
function x^e is zero. The values of the coefficients may therefore be 
obtained by a method which is similar to that used by the author, but 
which is somewhat simpler by direct proportional control of the valve 
of each input variable x i by the output of an integrator following a 
multiplier which has inputs x i and e. This method has the advantage 
that it may be extended to give an optimum linear expansion for every 
process output in terms of linear or non-linear functionals of the 
process inputs. 
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P. M. E. M. van der Grinten, in reply 

The method of ‘model updating’ as proposed by Dr. Barker is certainly 
a realistic alternative for finding process parameters (see also the paper 
by Blandhol and Balchen). It has the advantage of storing the in¬ 
formation and, in consequence, is capable of yielding faster results 
in certain cases. 

On the other hand, the influence of process dynamics, w hen 
measuring the static gain, remains the main difficulty; the value XfE 
therefore gives no information on how to move the model-coefficients 
A € . Maybe a combination of the method of my paper, that is the use 
of the moving averages of JQ and the method of the discussion, would 
be useful for this purpose. 
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EVENTUAL STABILITY 


The general objective is to select the control u so that the system 
is maintained near some desired state. It is customary and con¬ 
venient to take the desired state to be the origin x = 0 (zero 
error). The control u will in general be a function of the state x 
of the system and of time, and with the selection of .a specific 
control u eqn (1) becomes 

x~X(t,x,u(x,t)) = F(t, x) 

To be a satisfactory control it is necessary that in some sense 
the system be maintained near the origin. In other words, the 
origin is in some sense to be stable. 

It is accepted that stability or asymptotic stability means 
stability in the sense of Liapunov. There is today certainly no 
necessity for reviewing these definitions nor for describing 
Liapunov’s second or direct method 1 " 4 . There is, however, one 
point that should be made. The definition of stability or asymp¬ 
totic stability usually starts with the assumption that x = 0 is 
an equilibrium state; that is, jF(/, 0) = 0. In fact, this is inherent 
in the definition of Liapunov stability: without the assumption 
that the origin is an equilibrium state it can be shown that the 
definition itself implies that the origin is an equilibrium state. 
Thus Liapunov stability applies only to states which are equili¬ 
brium states. 

Now the kind of stabilities to be treated here are stabilities of 
states which are not necessarily equilibrium states but which as 
time passes tend to act more and more like stable or asymptoti¬ 
cally stable equilibrium states. If the adaptive control problem 
is considered with some degree of realism, then this is the kind of 
stability that one would expect to achieve. If the plant or system 
being controlled is subjected to perturbations and a changing 
environment, then it is best to have the adaptive control make 
the desired state behave more and more like a stable equilibrium 
state or, better still, like an asymptotically stable state. Such 
stabilities are called ‘eventual’ stabilities and it is thought that 
they may play an important role in the theory and design of 
adaptive control systems. 

In this paper concepts of eventual stability are defined, 
some of their basic properties are stated, and theorems which 
show how Liapunov’s direct method can be extended to the 
study of eventual stability are given. If the system being studied 
is autonomous or if the origin is an equilibrium state, then these 
eventual stabilities are the same as Liapunov stabilities. In these 
cases the theorems given here are generalizations of Liapunov s 
theorems, but when the system is non-autonomous and the origin 
is not an equilibrium state, there is something new. In addition, 
theorems are given which show how qualitative results on the 
degree of eventual stability may be obtained; that is, an eventual 
region of asymptotic stability can be estimated. An example is 
given illustrating how the theory can be applied. 

Eventual Stability 

The fundamental system is 

x=F(t, x) (2) 

and in some region Q it is assumed: t S 0, ||x|| < R, uniqueness 
of solutions and a continuous dependence of the solutions on the 
initial condition. Here ||x|| = (xf + x% + ... + x'D *, the 
Euclidean norm. When local properties are dealt with, it is al¬ 
ways assumed that one is in Q. When properties in the large 
are treated, R = °o. Let x(t,t 0 ,x°) denote the solution of eqn (2) 
that starts at time t 0 at x°; that is, x (t 0 ,t 0 ,x°) — x°. 


Definition 1 

The origin is said to be eventually stable if, given s > 0, 
there exist numbers d and T such that ||x°|| < <5 implies 
||x 0,r 0 ,x°)|| < e for all t ^ t 0 = T - 

This is then the precise statement of what is meant by saying 
that ‘as time goes on the origin tends to act more and more like 
a stable equilibrium state’. Eventual stability has also another 
intuitive meaning. If the origin is eventually stable, then the 
system has the property that if it has operated properly for a 
sufficiently long period of time it can be expected to continue 
to operate properly in the future. This interpretation of eventual 
stability is contained in the following result. 

Theorem I—Eventual stability of the origin is equivalent to 
the following: given s > 0 there exist numbers 6 and T such 
that ||x(Mo>*°)ll < & for some ST implies ||x (r,f 0 ,x°)|| < £ 
for all t Se t x . 

Eventual stability is closely related to Liapunov stability and, 
in fact, the following is true. 

Theorem 2 —If the system of eqn (2) is autonomous (F (f,x) 
= /(*)) or if the origin is an equilibrium state of eqn (2) (F(t,0) 
= 0), then eventual stability of the origin is equivalent to uni¬ 
form stability. 

It is, however, easy to show that there are systems where the 
origin is eventually stable but not stable in the sense of Liapu¬ 
nov. A trivial example is x = <p(t), <fi(t) > 0, J 0 <p(t) dt < °o. 

A theorem which is similar to Liapunov’s theorem on sta¬ 
bility is now described. Let V(t,x) be a real-valued function 
with continuous first partial derivatives on Q. Then V(t,x) is 
said to be positive definite on Q if 0 < u (||x||) V(t,x) for all 
(t,x) in Q, x 0. Let (j)(t) be a function with the property that 

f f' 0+fl ) 

lim sup -j sup 4>(t)dt?=0 (3) 

fo“ > °o = 0 J to ) 

This is, of course, a condition on the positive part of (j), (For 
example, if (j)(t)^0 and if Jo (0 t < co, then 4b (f) is a func¬ 
tion of this type.) If V(t,x) ^ (j)(t ) for all 0 ^ ||x|| < D, then 
it will be said that V(t,x) is asymptotically non-positive in Q. 
Define 

V( t ,x)=^H&™* x V)-F(t,x) 


dV 

a t 


+ E 


i =i 


dV 

6x ; 


F t (t, x) 


It is then not difficult to show that: 

Theorem 3 —Let V(t,x) be positive definite in Q and have the 
property that V(t,x) -*■ 0 as ||x|| -> 0, uniformly in t for f S 0. 
If V(t,x) is asymptotically non-positive, then the origin of 
eqn (2) is eventually stable. 


Eventual Asymptotic Stability 

As with eventual stability, eventual asymptotic stability is 
defined and an extension of Liapunov’s asymptotic stability 
theorem for this type of stability is obtained. 

Definition 2 

The origin of the system of eqn (2) will be said to be eventually 
asymptotically stable if (i) it is eventually stable and (if) there is 
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an r > 0 and a T 0 such that |[x°|| < r and / 0 ^ T 0 imply x(t 9 t 0i x°) 
0 as t -»■ oo. 

Condition (ii) states that if after a sufficiently long time the 
system is sufficiently close to the origin then it will tend to the 
origin as t approaches infinity. In analogy with Theorem 1 
there is 


Theorem 4— Condition (it) of Definition 2 is equivalent to: 
there is an r > 0 and a T 0 such that \\x (/ 1 ,/ 0j *°)|| < r for some 
h ^ To implies x (t 9 t 0 ,x°) -> 0 as / -> oo. Again as before: 


Theorem 5—If the system of eqn (2) is autonomous or if the 
origin is an equilibrium state of eqn (2), then eventual asymptotic 
stability of the origin is equivalent to uniform asymptotic 
stability. 

Turning now to the extension of Liapunov’s theorem on 
asymptotic stability, it will be said that V(t,x) is asymptotically 
negative definite on Q if V(t,x) g for r < ||x|| < R , where 
for each r > 0 /• 


<fi r ( t ) dt = — oo 


(4) 


o 


Theorem 6— Assume that the origin is eventually stable and 
that V(t,x) is positive definite. If V(t,x) is asymptotically nega¬ 
tive definite, then the origin is eventually asymptotically stable. 

In some respect this theorem is not as satisfactory as the 
theorem for Liapunov asymptotic stability. Here it is assumed 
to begin with that the origin is eventually stable and there is 
therefore the separate problem of establishing eventual stability. 
The reason for this is that what is called asymptotic negative 
definiteness does not, in general, imply asymptotic non-positive- 
ness. A more useful result is the following: 

Theorem 7—Assume that V(t 9 x) is positive definite on Q and 
that V(t 9 x) -> 0 uniformly in / for t ^ 0. If V(t 9 x) ^ - u(x) 
+ h(t) on Q, where u(x) is positive definite on Q and 


i 


e e s /z(s)ds-+0 as f-»oo 


(5) 


then the origin of the system of eqn (2) is eventually asymptoti- 
cally stable. 

It was pointed out by Bojanic that eqn (5) is equivalent to 

't + a 

h(s)ds=0 (5 a) 


1 


lim sup 
*->oo a >0 I + # 


This theorem states that if the system of eqn (8 a) approaches 
a uniformly asymptotically stable system sufficiently rapidly, 
then it is eventually asymptotically stable. Thus, for example, 
if x = f(x) has an asymptotically stable origin and F(t,x) ->f(x) 
uniformly in x for ||x|| ^ r 0 (some r 0 > 0) as t -> oo, then the 
origin of the system of eqn (2) is eventually asymptotically 
stable. Along these same lines there are also the following two 
theorems which give a method for obtaining qualitative in¬ 
formation on the nature of the eventual asymptotic stability. 

Theorem 9 —Let M be a bounded closed set of n dimensional 
vectors x which contains the origin. Let 77 be a subset of M 
with the property that solutions of eqn (2) which start in N 
at a time t 0 ^ T remain thereafter in M. Suppose that there can 
be constructed a real-valued function V(t 9 x) with the following 
properties: 

(a) V(t,x) v(x) as t -> co uniformly for x in M. 

(b) V(t,x) — w(x) as t -* oo uniformly for x in M. 

(c) v(x ) and w(x) are positive definite for x in M. 

Then there exists a T 0 > 0 with the property that x (t, / 0 ,x°) -~> 0 
as t -> oo for each x° in N and each t 0 ^ T 0 . 

Theorem 10 —Assume that the set M defined by v(x) s: l 
is bounded and that (a), (b), and (c) of Theorem 9 are satisfied. 
For each <5 > 0 let M d be the set defined by v(x) g /- d. Then 
corresponding to each <5 > 0 there is a T d such that x° in M d 
and t 0 ^ T d implies x (/,^ 0 ^ 0 ) 0 as / -> oo. 

Theorem 9 states that N is an approximate eventual stability 
region and its size gives a measure of the system’s stability. 
Theorem 10, which is probably simpler to apply, actually states 
that M 5 is an approximate stability region. As time goes on 
(S 0 ,T d ^ co) M 5 approaches M so that M itself can be con¬ 
sidered to be an approximate eventual stability region. 


An Application 

The example of adaptive control to be considered here is an 
extension of an idea due to Rang 6 . It is in some sense artificial 
but illustrates that if one has enough information (which might 
well be statistical) about the unknown functions in the system 
to be controlled (the plant) then adaptive control is possible. 
Use is made here of the following theorem, which is closely 
related to Theorem 10: 


In the application of this theorem it is useful to note (see eqn (5 a)) 
that condition (5) is satisfied if either. 

h(t)-+Q as tf-»oo (6) 

or 

fo ft (0 dt=c (c finite) (7) 

As a consequence of the converse theorems 5 on the existence 
of Liapunov functions and Theorem 7, the following result is 
obtained almost immediately. 

Theorem 8 —Assume that the system 

x = X(t,x) (8) 

has a uniformly asymptotically stable origin. Then the system 

x = X(t,x) + p(t,x) (8a) 

is eventually asymptotically stable if \p(t,x)\ ^ h(t) for 
\\ x \\ = ^o ( r o > 0), where h(t) satisfies eqn (5). 


Theorem 11 —For the system 

x =/ (t 9 x, y) (x an r vector) 

y = g(t ,x ,y ) (y an s vector) 

there is a scalar function V(x,y) with the following properties: 

(1) V(x,y) is positive definite and has continuous first partial 
derivatives for all x and y. 

(2) V(x 9 y) -> co as ||x|| 2 + ||y|| 2 -> oo. 

(3) V(x,y) SZ — fV(x) + h(t) q(x,y) where W(x) is positive 
definite and continuous for all x, q(x,y) is continuous for 
all x and y 9 and |/z(OI satisfies eqn (5) above. 

(4) x and y bounded imply f(t 9 x 9 y) is bounded for all t ^ 0. 

Then the origin x — 0, y = 0 is eventually stable and corre¬ 
sponding to each r > 0 there is a T 0 such that ||x(/ 0 )|| 2 + 
lbOo)ll 2 < f° r = F 0 implies that y(t) is bounded for t t 0 
and x(t) -> 0 as t ->■ oo. 
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In the example to be considered in a moment x is the error 
and y is related to certain control parameters. What is wanted, 
which is the conclusion of the above theorem, is after the system 
has operated for a while that the error x go to zero and the con¬ 
trol parameters remain bounded. 

The general nature of the adaptive control system is shown 
in Figure L The differential equation of the plant is 

x = oc(t)Ax + P(t)p (; t , x) +/ 


where c-j, Cq, are positive constants. Then 

V=-e'Cs + (k-l)(c 1 g l + k' 1 f'Qs) 

+ (£ — <x 0 ) (c 2 f 2 + s'Ce - y'A'Qs) 

+(»?-&>) (A g3-p'6 E ) 

+ (a —oCq) (y'A'Qs —s’Cs) 

+ (/?— Po) P S £ 


and the model is 

y — Ay + u (t) 

The states x and y are n vectors, A is a. constant n X n 
ma tri-x whose characteristic values all have negative real parts, 
and u (?) is continuous and bounded for all t S 0. For the plant 
it is assumed that p (t,x) is known and that it is bounded for all 
t ^ 0 and all x. All that is assumed to be known about the scalar 
functions «(/) and /3(f) is that they are continuous and have 
bounded derivatives 

a(f)->a 0 and p(t)->p 0 as f-co 



It is not assumed that oc 0 and /S 0 are known. The problem is 
to determine the function/so that the plant follows the model 
_ x ( t ) — y it) -*■ 0 as t -*■ oo) and the control parameters 
remain bounded. There are then good heuristic reasons for taking 

f = lc[(l-Z) Ax-ri p(t,x)+u] 
where k = gi (t,x,y,k,£,rj); £ = g a (t,x,y,kj,rj); and 
r\ =g 3 (t,x,y,k,g,rj). 

It is desired, of course, to determine the functions gi, g%, and 
g 3 . With £ = x — y the differential equation for e is 

e = A£ + (ot, 0 -£)Ax + ip 0 -n)P- + ( k - i -)k 


+ (a — a 0 ) Ax + (p — Po) P 

By assumptions on A it is known 1 that corresponding to each 
positive definite matrix C there is a positive definite matrix a 
with the property that A’Q + QA = - 2 C. (Here and throug - 
out the prime denotes transpose.) Select any positive definite 
matrix C and the corresponding Q and define 

2 F = £'Qe + c 1 (k-l) 2 + c 2 (£- a 0 ) 2 + c 3 (i? - Po) 2 , 


Selecting 

g^-c^k-'f'Qs 
g^-c^ie'Ca-y’A'Qe) 
g 3 =c^p'Qe 
the following is obtained 

V= - e'Ce+(oc-cc 0 ) (y'A'Qs—s'Ce)+(P~Po)p'Q c 

It is not difficult to see that all the conditions of Theorem 10 are 
satisfied. The conclusion then is that x = 0, k = 1, £ = <*o» 
rj = jS 0 is eventually stable and given r > 0 there is a T 0 > 0 
such that 

| |8 (f o) || 2 + (fc do) -1) 2 + « 0o) - «o) 2 + Of (to) - Po) 2 < r 2 

for any f 0 ^ T 0 implies that s(t) -> 0 as t -> co and kit), £ (f), 
*(,) are bounded for t 2s t». This, although not as conclusive an 
answer as might be desired, does say that in time the region in 
which satisfactory performance is obtained becomes the whole 
space. Without more information about the unknown functions 
which describe the plant this is perhaps the best that can be 
done. If a(t) = « 0 and pit) = Po, then all solutions 8(f). kit), 
£(t), r]{t) are bounded and each e(f) -*■ 0 as t -> co. 
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DISCUSSION 


R. Tarjan, Hungarian Academy of Sciences, Budapest, Hungary 


The author’s Definition 1 differs from the usual one (||x ||< & > || xt, 
^ iL.i xi_ » f/xv oil t ' v -» is rei 


Utnor S LlCmilUUil X umvm rw- - - " . 1 J 

t 0 , x° || < s for all t > t 0 ) in that the condition forall t > t 0 is replaced 


by the more general condition for all t 0 >t> T. 

Y The crucial point seems to be the time interval T , after which the 

origin a 0 tends to act as a stable state. 


In practical cases the salient point seems to be to establish more 
or less explicit conditions for the existence and magnitude of T This 
depends obviously on the parameters q of the system, which do n 
appear explicitly in eqn (2), the time dependence of which causes a 

drift of the origin x°. j 

Further, an adaptive control system differs from a system wi t 
time-varying coefficients in that some of these coefficients are con- 
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trolled in a suitable manner in response to a suitable reference signal, 
which may be derived from either the input or the output, or a model 
or a combination of them. In this case, however, as has been shown 
previously 1 , we have two control loops within the system: the main 
loop working in a conventional manner, and the adaptive loop 
controlling some of the coefficients (parameters) of the main loop. 
The notion of the eventual stability refers obviously to the main loop, 
neglecting the adaptive loop. If, on the other hand, we are considering 
both loops simultaneously, a generalized equilibrium can be defined, 
which may or may not be stable according to the well-known stability 
varieties. 

Reference 

1 Proceedings of the Symposium on Self-Adaptive Systems, Rome 
1962, in press 


B. S. Razumichin, Institute of Mechanics , Academy of Sciences 
Moscow , U.S.S.R. 

The problem of the stability of motion is the problem of defining the 
conditions under which some solution of the system of differential 
equations that adequately describe the motion under investigation 
corresponds to the real existing motion. Thus not every solution of 
the system of differential equations describes the real motion (that is, 
the motion that can be realized). In this paper the definition of stability 
of some sets of the system states is described, but the set of states is 
not even a solution for the system of equations [Fit, 0) =j= 0]. For this 
reason I cannot agree that the stability definition given in the paper 
is a generalization of Liapunov’s definition as the eventual stability 
is a development of Liapunov’s definition for the class of fictional 
motions, which gives it a metaphysical character. The paper mentions 
some extremely important problems and contains proofs of theorems 
the correctness of which cannot be doubted. It seems to me, however 
possible to set and solve the problems of the given class by the methods 
of the classical theory of Liapunov. For this purpose I can propose 
two methods. 

(1) From the authors’ definition it follows that the system of 
differential equations under investigation is such that there is one 
solution xft) of the system, which approaches infinitely the zero point 
of coordinates, that is, such that for every e > 0 it is possible to 
determine a T> 0 such that ||*(01| < s with t > T. This is a real 
solution and the eventual stability is equivalent to the statement of the 
fact that there also exists the usual Liapunov’s stability of such a 
solution. 

(2) The system (1) considered in the paper 

x^Fi (t, x) ( 1 ) 

can be represented in the form 


be recommended in cases when it is comparatively easy to find the 
solution which satisfies the condition x t (t f) —> 0. The second method 
can be recommended for all other cases. 

J. P. LaSalle, in reply 

The remarks that Dr. Razumichin has made are quite interesting 
although I must confess I am unable to understand his objections to 
considering eventual stability as a generalization of Liapunov stability. 
His inference that eventual stability is a ‘development of Liapunov’s 
definition for a class of fictional motions which gives it a metaphysical 
character’ and his first proposal for overcoming his objections indicate 
that there is a misunderstanding. 

Eventual stability of a state of a system is a simple and natural 
extension of Liapunov stability of a state. They both have to do with 
the asymptotic properties of solutions (motions) which start near the 
state. It is true that the motions are not the real physical motions of 
the system but the motions of the state of the system in state (phase) 
space. In this respect eventual stability is similar to Liapunov stability 
and, in fact, eventual stability includes as a special case Liapunov 
stability (Theorems 2 and 5). With the exception of some details of 
proofs and the generalization of conditions, the methods and concepts 
do not differ greatly from those of classical Liapunov theory. 

In his second proposal, Dr. Razumichin brings out a point with 
greater clarity than we did in our paper. What he says is that if 
h(t)= lF(t,0)l satisfies eqn (5) and if the origin of the system & «= 
F(t f x) — F(t,0) is uniformly asymptotically stable then the origin 
of the system * = F(t,x) is eventually asymptotically stable. This is 
the significance of Theorem 8: eventual asymptotic stability can be 
investigated by classical methods. Dr. Razumichin is also correct 
when he points out that Theorem 8 is closely related to total stability 
(stability under constantly acting disturbances). 

It seems worth pointing out also that Theorem 11 is related to the 
important problem of the study of the stability of manifolds. It is a 
result of the eventual asymptotic stability of a noncompact manifold. 
Here the noncompact manifold is the r-dimensional linear subspace 
x = 0, which in applications to control systems corresponds to zero 
error. Considerations of how this may be applied suggest the need for 
a stronger and improved version of Theorem 11. With the same 
notation as in Theorem 11 a new result in this direction is the following. 
Consider the system 

% — fit, x, y) (x an r vector) 

(*) y ~ git , x, y) (j an s vector) 

where/0, x, y) is bounded for bounded x and y and all / ^ 0, and the 
following conditions on a Liapunov function Vix,y): 

(1) Vix, y) is positive definite and has continuous first partial 
derivatives 


where *i-*&*)+F, (f, 0) (2) 

0)=0 and F t (t, 0)~+0 (3) 

t-*oo 

Jkh,rK S a tem (1 l°T b ® f nsidered as a astern with a constant acting 
disturbance which satisfies the conditions (3). The same problem was 

onsidered by I. G. Malkin and others in the limits of Liapunov’s 

sSt°em'rn OT H ltl0 .K 3 PermitS proof of the asymptotic stability of 
i i^ U . n< ^ er assum P tl0n of uniform asymptotic stability of the 

em a 8 l°fT n ° f w 1 ™ * = *) (this corresponds to Then! 

nf thl J the Iiaper). I n conclusion it may be said that the problem 
of the stability of the system infinitely approaching the equilibrium 
state can be solved in the limits of the classical Liapunov’s theory and 
two ways are mentioned above to solve this problem. The first may 


W rvc,y)-+ co as pc\\* + ||y|| 2 ->oo 

(3) V(x, >) £ — W(x) + hjt) q(x, y) + h 2 (t) V(x, y) 
where (i) W(x) is continuous and positive definite 

0*0 vix, y) is continuous 
(«7)]‘“ |VO|dt<oo, i = 1,2. 

TAeown— If for the system (*) there exists a function V(x, v) 
satisfying (l)-(3), then the state x = 0, y = 0 is eventually stable and 
t0 each r >° there is a Tr such that jjx(tfj) |p + 
r for some ? o ^ T r implies y(t) is bounded and x(t)-> 0 

If, in addition, 

(4) For some K > 0 and some 0 < a < 1 

y)\ ^ KV«(x, y) 

then all solutions y(t) are bounded and all x(t) 0 as t -> oo. 
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Non-linear Stability Analysis for Restricted Non-linearities 
Using the Second Method of Liapunov 

H. NOUR ELDIN 


Summary 

The methods treating the stability of a closed loop with single non¬ 
linearity lead to stability conditions that assure stability regardless of 
the shape of the non-linearity within a section in the non-linear plane. 
Thus, the stability criteria are always more than sufficient conditions 
for non-linearities that accept some restrictions on its shape. These 
restrictions must cause relaxation of the stability conditions. In this 
paper, this approach is used for both inherently stable and unstable 
control systems to get stability criteria that take into consideration the 
restrictions on the shape of the non-linearity. 


absence of continuous external disturbing forces. The transfer 
function of the linear part (Figure 1) will be G 0). 

The linearized closed-loop system (y = k x), may be one 
of the two types: 

(a) Inherently stable control system which is stable for any 
value of gain. 

( b ) Inherently unstable, which is stable in a certain range of gain. 


Sommaire 

Les methodes qui traitent la stabilite d’une boucle fermee contenant 
une seule non-linearite conduisent k des conditions qui assurent la 
stabilite dans un secteur du plan non-lineaire sans considerer la forme 
de la non-linearite. Ainsi, les criteres de stability donnent toujours des 
conditions plus que suffisantes si les non-lindarites acceptent certaines 
restrictions pour leur forme. Ces restrictions permettent de relacher les 
conditions de stabilite jusqu’a un certain point. Dans ce rapport ces 
considerations sont utilisees pour obtenir des criteres pour la stabilite 
de systemes dont la fonction de transfert est stable ou instable en 
prenant en consideration des restrictions de la forme des non-lindarites. 

Zusammenfassung 

Die Methoden, welche die Stabilitat einer geschlossenen Schleife mit 
einer Nichtlinearitat beschreiben, fiihren auf Stabilitatsbedingungen, 
die die Stabilitat in einem Sektor der nicht-linearen Ebene garantieren, 
unabhangig von der Form der Nichtlinearitat. Deswegen sind Stabili- 
tatskriterien immer mehr als hinreichende Bedingungen, wenn die 
Nichtlinearitaten gewissen Beschrankungen in ihrer Form unterwor- 
fen sind. Diese Beschrankungen erlauben eine Milderung der Stabili¬ 
tatsbedingungen. In obigem Artikel wird diese Betrachtungsweise an- 
gewendet, um fur Regelungssysteme, deren offener Kreis stabil Oder 
unstabil ist, Stabilitatskriterien zu finden, welche die Beschrankungen 
in der Form der Nichtlinearitat beriicksichtigen. 


Introduction 

The stability analysis presented in this paper deals with a single 
loop with single non-linearity. The block diagram of Figure 1 
shows the closed loop, for which the stability is studied in 



Figure 1. Block diagram of a single loop with single non-linearity 
without input function 

Block L The non-linear element y = f(x) Block 2. G (s) = G x (s) G 2 (s) 


Inherently Stable Control System 

In this system, the open-loop poles and zeros of G (s) are in 
the left half of 5 plane. If the poles are simple, and their number 
is more than the number of zeros, G (s) can be expanded in 
partial fraction form. 


G(s) = 


-X(s) 

Y(s ) 


n+ 1 


- E 

T = 1 


$ +Aj 


where L = /(x), = pole of G (s) (real or in complex pairs, 

but different), = residues of G (s) at the corresponding 
pole, and n = order of the transfer function. 

In this case, the transfer function G (s) can be replaced by n 
first-order transfer functions in parallel. The output variable 
after each first-order link z* is related to / (x) by the relation 


d t 


-XiZi+fix) 


and to x by the relation 

n + 1 

X= Z «; z ; 

i=l 


Zi is called the canonical variable. 

The system equations can be written now in either one of 
the canonical forms 



Z;+/(x) i=l>2 ,n 

at 


and 

n 



II 

M 

~i 

(la) 


i= 1 


or 

j n+1 

^=-A iZi +/0), x=Ea jZj 
at i =i 

(lb) 

where 

y=f(x ) 



n 


ft=-o Wt> f, = -Z a i 

i = l 

One can get a simplified stability criterion using Lurie stability 
theorem. There are different simplified stability criteria 1 , in 
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which no conditions are imposed on the shape of v = f(x) 
except JK } 


V is negative definite if 


= J o /(x)dx>0 or xf(x)>0 for all |x|>0 

This means that the non-linearity is mainly in the first and third 
quadrant of [x, /(x)] plane, and passes through the origin. 

There is no other restriction on the shape of /(x). That is 
why the simplified stability criterion is more than a sufficient con¬ 
dition. One can relax the stability conditions by taking the shape 
of the non-linearity/(x) into consideration. 

Consider as a Liapunov function 

v= ,,]± i ZfZj ' +x2//2 a ‘ J = dji ® 

which is positive definite everywhere for simple roots. 

V=Z'(z i ,x)BZ(z i ,x) 


|a £j [ >0 and 


A -6 
-0 4 rf, 


-e l o 

—-... -rK 

2 2 


V is negative definite if |a ; ,|>0 and A 6 >0 (3i 

— Q4rk v } 

^ = II ^ij If s 0 = 01 , 9 2 , .. 6 n 


Z lj^X, K+ ^ e ‘ 


z %J^+x l +Ml ~ e ' < 7 > 

i = l,2 ,..., n 

The general non-linearity f(x) must pass through the origin, and 
its slope has always to satisfy the above stability condition. 

One can obtain a combination of the two criteria by adding 
the Liapunov functions. The new stability conditions will give 
restrictions on both K and f^x). 

The condition that fix ) must pass through the origin can be 
relaxed if we consider as Liapunov function 

V= j.,i^ij Z,Zj+ T [/(5,)+£ f 

which is positive definite if /(0) + £ = 0. £ is a constant. 

The above stability conditions lead to the expansion of 
determinants of the order «4-l. Similar stability conditions are 
derived which reduce the stability question to the expansion 
of n/2 independent third order determinants. 

Generally one can obtain, for a certain G(s), the sufficient 
restrictions on the shape of the non-linear element that insure 
stability conditions. 

If we use as Liapunov function 


V=F(z^x 2 / 2+1 X (A(x)) 2 


where. 


^ k =o 
I=/oW 


the general non-linearity 

f K (x) = d K f(x)/dx K 


y- J W , , T 

K —— 1 = 1,2,...,n 

In this criterion, the factor f(x)/x appears. One can find the 
range in which f(x)/x can satisfy the stability condition. 

Again, consider the following Liapunov function 

F= i ,| 1 I^t Z ^ + t/W] 2 /2 (5) 

which is positive definite if /(0) = 0. fix) may be in any of the 
four quadrants. 

V=Z' (z h f(x)) BZ (z ; ,f(x)) 


i J± 1 X i +Xj ZiZi a V~ a J 



= £ <Ka;V«; + iZi +1 ) 

i— 1 

a ‘ z ^+ 

a t+i z i+i z i+i j a Ui+lZi Zi +1 

A+/t t 

A i + i+Xi + i X 

1=1,3, 

1 


In Appendix I, some stability conditions are shown for different 
restrictions on the shape of the general non-linearity/(x). 


=k -ZljLyf 
2 2 Jl 


f _ d /W 
h ~~&r (6) 


Inherently Unstable Control System 

The study previously made for these systems introduces 
restriction on the amplification factor /(x)/x. Simplified stability 
criteria can be obtained either by pole or zero shifting 1 , or by 
using certain Liapunov functions that lead to stability determina¬ 
tion if /(x)/x lies in a certain region. Generally, the linearized 
system is stable within the hatched area shown in Figure (2), and 
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RESTRICTED NON-LINEARITIES USING THE SECOND METHOD OF LIAPUNOV 


only the cross-hatched area is stable for a general non-linearity 
having no restrictions except that it lies in this cross-hatched 
area. If more restrictions are imposed on the shape of f(pc), one 
can get stability criteria that extend the study of the system to 
the linearized stability region. The technique shown before can 
be applied for inherently unstable systems in conjunction with 
the pole-zero shifting method 1 , or by selection of certain 
Liapunov functions. 



Figure 2. Region of stability for non-linear and linearized control system 
1 , For linearized system 2. For non-linear operation 


Pole and Zero Shifting Technique 
The system equations are 
y=f(x ) 


TO=- 




TO 


E &«s 


i = 0 


m<n 


If GO) has poles in the right-hand side of the s plane, one uses 
pole shifting by substitution 


For a pole shifted system, one can use the stability condi¬ 
tions in Appendix I. In this case the restrictions in these crite¬ 
ria will apply for g (x ). The constants 1, a, /3, r are those of 
the shifted transfer function G'O) given by eqn (10). 

In a zero shifted system, the number of poles in G' 0) are 
equal to the number of zeros. One has to use the modified first 
canonical form 1 


and 


or 


fe—V,+/(»■) 

(13 a) 

x'= E a i z i -cj(.x’) 

1=1 

(13 b) 

i = l ut 

(13c) 

)'%' — X Pi z i~~ r J ( X ) 1 = 1,2, .. 

,,n 


i= 1 


The values X, x, P, r are those of the original transfer function 

G(s). . . 

One can get stability conditions using the modified canonical 
form. However, this is not necessary as we can get the range of 
stability without using zero shifting. 


Arbitrary Choice of Liapunov Function 

The methods used before depend basically on the canonical 
form which requires the determination of the roots and residues. 
In the following, the general non-linearity is transferred directly 
in the system equations and one chooses the Liapunov function 
as if the non-linearity were already given in mathematical 
form. 


Example 1 


TO = 


6 


s + 3 

s 2 + 2s-l 


(14) 


This system was shown stable for general non-linearity 1 only if 


Y'=f(x)-c p X = g(x)... 


(9) |/(x)|>|6x| for all |x|>0,/(0)=0 


<TO = 


TO) 

TO 


E “d 


£ —0 


n rn 

E b i s i + c p E a/ 

i — 0 i -0 


( 10 ) 


If GO) has zeros in the right-hand side of the s plane, we use 


zero shifting by substitution. 

X’ = X-c z Y 

m n 

(11) 

o-W-- r «- 

Y(s) 

E a/+c z E b/ 

i = 0 i~ 0 

E b/ 

i = 0 

(12) 


The values of c$ and c z are chosen such that all poles and zeros 
are shifted to the negative half plane, which is the condition for 
linear stability. 


The system equations can be written in the form 
x x = x 

Xi — X2 

x 2 = TOTO*)/6)* 2 -(K/2-1 )*i C 15 ) 

let 

V=ex\l2+xll2 for s>0 (16) 

d7/dt = (8-K/2 + l)* 1 x 2 -(2+/ 1 (x)/6)*| 

Condition of stability will be 

1 — K/2 + £ = 0 /(*)/*> 2 | 

and 2 +/i (x)/6 >0 4 f (x)/dx > -12 J 

This shows that one can get a restricted non-linearity which 
covers all the stable region of the linearized system. 
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Systematic Choice of Liapunov Function 

Ingwerson 2 , has developed a powerful systematic attack to 
get a Liapunov function from the system differential equations. 
It is interesting here to get the conditions of stability of a second- 
order system using his systematic technique. 

Let 

rz(c\— ___ __ a i s + a 0 


region of the linearized system. Using Ingwersoffs method and 
writing the non-linearity with its derivatives in the system equa¬ 
tions, one gets stability conditions for a higher order of <7(». 

Stabilization of Non-linear Control 

If one considers the second-order system in which 


Y s 2 + b i s+b 0 


y=f(x) 


The system equation can be written in the form 

x + (b 1 + a i f 1 (x))x + (b o + a o K)x==0 (18) 

Using the Ingwerson systematic technique to get ^(Appendix III) 


V — b 0 xl/2+a 0 J f (x 1 )dx 1 +xl/2 
which is positive definite if /(0) = 0 and 


therefore 


&o*i/2 + fl 0 J o fiixJdX^O 


b o + a Q ^~^>0 


Example (2) 


G(s)=- 


/(*) O+i) 2 


This system was solved using pole and zero shifting 1 , it is 
stable if 

|0-5xj<|/(x)|^|x| for all |x|>0,/(0)=0 
Rewriting the differential equation we get 

*+[2+/i(x)]x+(l--K)x=0 
The system is stable if 


1-/ (x)/x > 0 
2+d/ (x)/dx>0 


l/WI^N 

d/(x)/dx> —2 


This shows that there is no limit of /(*) so long as df(x)/dx^> 

o°ZTr ^ by theSC stabUit y conditions for the second- 
y em one gets a stability region that covers all the stable 


dV/dt=-(b 1 +aJ 1 (x))x 2 2 
which is always negative if 

b i+ a ifi(x)>0 

So, the conditions of stability of a non-linear second-order 
system with single non-linearity is that the coefficients of x and x 
of equation (18) are always positive. 

Applying this result to example (1), one gets the conditions 

f(x) 

Y^-^OmdlFf.ix^O 

which are the conditions (17). 


l>o + a 0 f(x)/x<Q 

one can get means for stabilizing it. Again considering the 
Liapunov function 

V=b 0 xl/2 + a 0 j f (xj) dxj+xf/2 

If relay characteristics are introduced for x such that 
x 2 =M sat (x) 

then the first condition for stability will be 

b 0 x\ + a 0 \ f (x) dx t > — M 2 1 2 
or y (19) 

&o + ^o(/ (x)/x)> —(M/x) 2 

The system will be regionally stable in the region of x in which 
the above inequality applies. Thus, we have a stability condition 
that allows the system to operate in a region of the [/(x), x]- 
plane within which the linearized control is impossible. 

Conclusion 

By imposing restrictions on the shape of a non-linearity in a 
single-loop control system, one can get sufficient stability con¬ 
ditions. The possible restrictions allowed to be defined determine 
generally the way in which the stability conditions are chosen. 
Two ways are described to get sufficient stability conditions 
and some of the conditioned stability criteria for inherently 
stable or unstable control systems are given. 

Appendix I 

Conditioned stability criteria for systems described by the 

first canonical form of differential equations K — *— and 
/i(x) = d/(x)/dx x 

Re k> 0 M0! ,0 2 ,...,0 b ) 




a ij~ a ij i 


/=1,2, ...,n 


Stability equation 


t~t~' 


L jif K+ ^=° 


*'==3,2, 


for all ijcl > 0 


l “" l>0 -» 4rk >0 


’W* 


for all |x| > 0 
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If + If 


+/l/?f 


-2 i—1, 2, ...,n 

d f >0 


One gets a simpler inequality 


4r/ t 


which leads to <p 


2 K 

for all |x|>0 


/=1,2,3, ...,n 

|- 9 4 “ r /j >o 

+A+i=0i+i 


If+If 


+/iA' 


-2 


r/i ciij 


<0 


The inequalities in Appendix I are applicable when x is a 
coordinate in the canonical form. If x is only an algebraic 
equation we use 


<0 


a i +1 , a i,i +1 

Llf+l-HI/+1 Ij + If+lJ 


i = l,3,5, — 1 n even 


tff>0, 


a i a i,i +1 


>o, 


fl f a f, Z+l 

^f,f+l ^f + l 


a i | a i,i +1 


Llf + If If + If+1. 




£7 


“0£ “^i+l “ 

+/l/?i = 0£ , 

~k/lA+l = 0f + l 


-A* 

“®i + l 

8 


-rk 


>0 


t, t +1 


[_If + 1 ~h If + 1 Ifhlf+l 

i= 1,2,3,w —1 n even 


^f>0, 


f+1 

^ 1 , i+1 ^i+1 


> 0 , 


^f a £, i+1 @i 

®i,i+1 a i +1 ^i+1 

-ft -ft+i -^-Vi 


>0 


Appendix II 

The conditions for — /? to be positive definite are 

A -0 


|flyl>0. 


-0 4r/ t 


>0 





A 

-9 

a 


a ik a k 

<0, 

-e 

Ark 

-1 


... a r 0 


a 

-1 

0 

kJ& k- 

X 

1,2,..., 

n , 

a = (a l5 

a 2 ,. 


Appendix III 

i =/(*) 

Xi=X 

x t =x 2 

x 2 = ~[b 1 +a 1 f 1 (x)~\x 2 —(b 0 + a 0 K) x t 
x =B(x)x 

0 l 

-[^ 0 + «o/l(^l) + «l/2(^l)^2] “ [^l+ fl l/l( X l)l 

with B'A + AB=—C 
C=2[b 1 + a 1 f 1 (x 1 )] 


B = 


A = 


&0 + «o/l( X l) + «l/2( X l) X 2 0 


.0 


^(Xi,X 2 ) = 

by integrating 


b 0 H" fl 0 /i C*T) 

0 


VF=b 0 *l+«o/( X l) + *2 

integrating one gets 


The values of can be chosen to satisfy \a^\ > 0. The second 
condition when expanded will take the form 




f(x 2 ) dx x + 


x\ 


D t (a v )-fl + D 2 (a l7 )/i + £3 (fly)>0 


The above inequality will give a range of real f 2 (for constant 
a is ) as long as Z> 2 2 — 4 D 2 D 3 > 0. By varying one of a i3 we 
get from the set the range in which the non-linear system is 
asymptotically stable. 

If a i3 = 0 for i # j the inequality reduces to 


i=l U ij 


Nomenclature 

x Input variable to the non-linear element 

X Input function to the non-linear element 

y Output variable from the non-linear element 
Y Output function from the non-linear element 
z The canonical variable 
/ (x) The non-linear relation y = /(x) 

G 0) Linear transfer function 
5 Complex variable 
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Notation for residue 
2 Notation for pole 

i Index order 

f Index order 

a Arbitrary constant 

n Order of the system transfer function 

£ Constant 

K f(x)/x the linearized gain 

f K (x) d K f(x)/dx K 
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DISCUSSION 


G. P. Szego, Research Institute of Advanced Studies , Baltimore , 

M.D ., U.S.A. 

Between the stability equations obtained by the author, and those 
obtained by Lur’e, Letov and Rekasius and summarized by Lur’e 1 , 
there exists a major crucial difference. The stability equations obtained 
by the above-mentioned authors are indeed sufficient conditions for 
the absolute stability 1 of the system (1); as such, these conditions 
depend only upon the properties of the linear part of the system (1). 

On the other hand, the stability equations presented in this paper 
contain explicitly the functions 


K(x) = 


/(*) 


and f 1 (x) = 


df 

dx 


and as such may be generally satisfied only by one particular non¬ 
linear system. However, in the case in which the non-linearity f(x) 
is known there exist methods 2 “ 5 which allow a systematic construc¬ 
tion of ‘global Liapunov functions’, i.e. Liapunov functions which 
describe the stability properties of any system in the whole state space. 
Even more severe shortcomings of this work are revealed by eqns (3) 
and (4), which can be satisfied only if K and ffx) are constant, i.e. for 
linear systems. 

The problem of absolute stability of the system (1) with respect 
to certain restricted non-linearities is, however, very important. 
I shall solve it following the procedure of Kalman 6 . 

Consider the control system £ 


(i) 


( 2 ) 


x=Ax — acp((j) 
a=2b'x 

and the continuous non-linear scalar function (p: 

(p{ 0)=0 0<cp(a)^(T 2 k k< + oo 

In (1) A is an n X n matrix, a and b are vectors. It is assumed that 
the matrix A is stable, that all linear systems in the class (1) are 
asymptotically stable and that the linear part of 2 is completely 
controllable and completely observable 6 . 

We shall then represent S in its normal form 5 . 

Consider the scalar function 


(3) 


v — x'Hx+fi J cp(s)ds h> 0 

whose total time derivative along the solutions of system (2) is 

V=x’ [. A'H+HA ] x-2(px'[Ha-pA'b-ab] 


-cp 


<x~+2pb'a 


-otcp 


a- T cp 


where the identity has been used 


-a \jpa—2 (px r b~\ — C 


(4) 


(5) 


Let 


A'H-\-HA = — qq' 
Ha — (ft A' + a) b + yq 

aL + 2jSfo'a=y 2 >0 


( 6 ) 


If there exist a real scalar y, a real vector q and a real matrix 
H — H' which satisfy the conditions (6), the scalar function (4) 
will take the form: 


V~—[x'q + (pyf 


1 

~a a —- 
k 


(p 


(7) 


By using the theorem proved by Kalman 6 one can show that for 
the existence of y, q and H satisfying eqn (6) it is necessary and 
sufficient that the conditions 


-+2fib'a + 2Re {(fib' A + ocb') (Lico — A) 


k 


+2/3b'a>0 


( 8 ) 


(9) 


are satisfied for all real oo. 

The condition (8) may be rewritten in terms of the open-loop 
transfer function G(m) of the linear part of £ as follows 


- + Re {(/? ico + a) G (ico)} ^ 0 


( 10 ) 


Theorem- If (9) and (10) are satisfied for all real co and some 
real a 0 and p, the system (1) is absolutely stable [globally asymp¬ 
totically stable for all non-linearities of the class (2)]. 

Proof. From Kalman 6 it follows that for a ^ 0 the expression (7) 
is negative definite along the solutions of the system (1). To complete 
the proof one must show that v is positive definite. Since x'Hx > 0 

value of ° ^ triVially tme ' Let °* Considcr thc minimum 

,J # <P(s)ds 

fnr^^t (2 l WhiC : h iS /? r 9?(CT) = ka ' U haS been aSSUmed that 

SS2 ® “ "»“»">■ •“■>". i»»«»»"■» 

One can similarly operate in order to take into account the 
s”“on“ e ° f the n ° n - linearit y ^ (Ezeilo 7 ) into the 

the basic inequality (S) bC treated m a S3miIar way and 

References 

1 LV c2’fi LcfT/ N ° n ~ li y ar Pr °blems in Theory of Automatic 

Control. 1951. Moscow; Gostekhizdat 

2 Z Z V 1957 L T ^ Me k°f ° f A ' M - UClPUn0V aHd Their A PP ,ica - 

tion. 1957. Leningrad; Isd-ro L.G.U. Leningrad 


566 



NON-LINEAR STABILITY ANALYSIS FOR RESTRICTED NON-LINEARITIES USING THE SECOND METHOD OF LIAPUNOV 


3 Ingwerson, D. R. Trans. Inst. Radio Engrs. N.Y. PGAC, 6 (1961), 

199-201 

4 Szego, G. P. Trans. Amer. Soc. mech. Engrs. (D), 84(1962), 571-578 

5 Szego, G. P. Trans. Amer. Soc. mech. Engrs. (D),85 (1963), 137-142 

6 Kalman, R. E. Proc. Nat. Acad. Sci. 49 (1963), 201-205 

7 Ezeilo, J. O. C. J. London Math. Soc. 37 (1962), 405-409 


J. C. Gille, Ecole Nationale Superieure de VAeronautique, 32 Boulevard 
Victor , Paris, France 

The result indicated by the author on the stability of the second-order 
equation with non-linear dissipation term (in x) (18), may be general¬ 
ized. Consider the more general non-linear equation of second order 1 

xf 1 (x)+xf 2 (x,x,x)+xf 3 (x)=0 1) 

and take the Liapunov function 

K(x,x) = j* f t (x) dx +dx (2) 

It is quite easily found that the derivative is 

V(x , x)= ~/ 2 (x,x, x)x 2 


The condition of single-valuedness of the characteristics of the 
non-linear element gives an explanation of the well-known phenome¬ 
non that a system composed of a linear element of second order and 
of a relay without hysteresis, is always stable, whilst the same system, 
but with the relay having hysteresis, may be unstable. 

By virtue of the Liapunov function this is evident since the charac¬ 
teristic f(e) of a relay without hysteresis is single-valued and, con¬ 
sequently, the integral * g 


/(e) de 


0 


is always positive, whilst the characteristic/^) of a relay with hysteresis 
is not single-valued and the integral may be negative for certain 
values of e as shown in Figure B. 

Liapunov functions of type (2) may be generalized for certain 
equations of higher order. In this way, we may prove the stability of 
the equation of third order of the type 


x + fi (x, x, x) x + ax +f 2 (x, x, x)x = 0 


if/i and f % are positive. 

Reference 

1 Gille, J. C. and Wegrzyn, S. Sufficient condition for the stability 
of non-linear systems of second order. Automation (November 1963) 


which is never positive if 

f 2 (x,x i x)> fi>0 

Further, the function V(x, x) being the sum of two integrals, is always 
positive if functions ffx) and ffx) are positive and if, moreover, they 
are single-valued, since an integral 

/(x )dx 
J o 

may be negative if f(x) is positive but not single-valued ( Figure A). 
There is, in consequence, an unlimited stability for eqn (1) if (l)/ 2 is 
positive and (2 )f x and / 3 are positive and single-valued. 



M. Aizerman, Institute of Automatics and Telemechanics , Moscow , 

U.S.S.R. 

I cannot understand this paper. The equations being solved [for 
example eqn (3)] serve to determine a* coefficients of the quadratic 
form, but these equations have the term k = f(x)/x. For this reason 
their solutions do not give numbers but functions afx). However, in 
this case the Liapunov function (2) is not a quadratic form and it is 
not clear why the existence of real functions satisfying eqns (3) serves 
a definite, positive Liapunov’s function (2). This remark leaves the 
rest of the paper not clear. In the best case we are in front of a hypo¬ 
thesis if eqns (3) have real solutions by any Kj <C K < then an 
absolute stability exists. 

Unfortunately this paper does not contain the proofs and offers 
no reason in favour of this hypothesis. 


H. Nour-Eldin, in reply 

I thank the discussors for their valuable remarks. I have added 
Appendix II to show that it is possible to obtain a range of asymptotic 
stability. I agree that by taking 6 = 0, (0 X = 0 2 = ... = Q n = 0) the 
inequality 


will lead to a system of equations and we cannot get a range of 
asymptotic stability in this case. I have already omitted such equations 
from the text. 


\A 6 
0 Ark 


>0 
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The Use of the Technique of Linear Bounds for Applying 
the Direct Method of Liapunov to a Class of Non-linear 

and Time-varying Systems 

R. A. NESBIT 


Summary 


By making use of certain inequalities, the stability of a class of systems 
is shown by the use of the basic theorems of the direct method. This set 
of inequalities defines a region, and if the non-linear or time-varying 
part of the system remains inside this region, then the stability of the 
system is assured. The usual methods of approximating the time 
response may be used, and thus this technique can be used to aid the 
design of control systems. The technique is applied to second- and 
third-order examples and is compared to the Variable gradient’ method 
of solving these problems. 

Sommaire 

En utilisant certaines inegalites, ce rapport montre comment deter¬ 
miner la stability d’une categorie donnee de systemes de reglage au 
moyen de la «methode directed L’ensemble de ces inegalites definit 
une certaine zone, et si la partie non-lineaire ou variant avec le temps 
se trouve a l’interieur de cette zone, la stability du systeme est alors 
assuree. Les methodes classiques d’approximation de la reponse en 
fonction du temps, peuvent etre utilisees et completes par cette nou- 
velle methode, en vue de la determination des systemes de reglage. Cette 
methode est appliquee a des exemples du 2e et du 3e ordre, et comparee 
aux resultats auxquels conduit l’application de la methode des gra- 
dients variables aux problemes de ce genre. 

Zusammenfassung 

Durch Anwendung bestimmter Ungleichungen kann man die Stabilitat 
emer Klasse von Systemen mittels der grundlegenden Theoreme der 
direkten Methode von Ljapunow priifen. Diese Ungleichungen defi- 
meren einen Bereich; befxndet sich der nichtlineare oder zeitverander- 
liche Teil des Systems innerhalb dieses Bereiches, so ist die Stabilitat 
dieses Systems gesichert. Dabei kann man die ublichen Methoden zur 
Annaherung des Zeitverhaltens benutzen, wodurch sich dieses Ver- 
fahren fur den Entwurf von Regelsystemen eignet. Dieses Verfahren 
wird auf Beispiele fur Systeme zweiter und dritter Ordnung angewen- 
aet und mit der „variablen Gradienten-Methode 44 zur Losung dieser 
Probleme verglichen. 


Introduction 

The direct approach to the study of the stability of motion which 
stems from the work of Liapunov 2 has been considered, pre¬ 
sented and generalized in a large number of works, mainly by 
Russian and German investigators. Over 150 different references 
by 98 authors, published before 1958, are given by Hahn 1 . 
Of these works only about 11 were then available in English. 
At the end of 1959, Kalman and Bertram 3 " 4 summarized the 
theory for English speaking readers. 

Also, because of the growing interest in this general theory 
of non-linear systems many of the important Russian works are 


now available in English 5 as well as many papers and some books 
written in English 6 . The principal works include those of Aizerman, 
Barbashin, Chetaev, Erugin, Hahn, Krasovski, Letov, Liapunov’ 
Lurie, Malkin, Schultz and Zubov. 

The direct method encompasses a number of theorems 
centred around so-called ‘Liapunov functions’ which are 
generalized ‘distance’ or ‘energy’ functions and around some 
precise definitions of stability. From a mathematical point of 
view the theory is both general and relatively complete. It is 
known that if a system is stable then there are many suitable 
Liapunov functions to prove stability, and the conditions on the 
Liapunov function for stability are concisely known. From the 
point of view of application of the theory to systems of interest 
to the control engineer, much has been done and much remains 
to be done. There are large classes of systems which are non¬ 
linear and/or time varying for which the stability behaviour 
can be determined by carrying out computations which have 
been prescribed already. Thus, it is worthwhile to study the 
canonical problems and their solutions (see the works of Lurie 
Aizerman, and Letov). 

However, until recently the application of the direct method 
has been impeded by the absence of a technique for constructing 
Liapunov functions. One of the first methods for constructing 
these functions is due to Zubov (see Hahn 1 ) which requires 
solving a linear partial differential equation involving the 
gradient of the function. Recently Schultz 7 proposed a ‘variable 
gradient method’ of constructing Liapunov functions; this 
technique is based upon assuming the form of the gradient, VF, 
and requiring single valuedness of V. The requirement that V 
be single valued (equivalent to V X V F = 0) allows the integra¬ 
tion of the gradient along any convenient path to obtain the 
Liapunov function from the gradient. This construction method 
allows the application of the second method to many problems. 

An alternative approach to assuming the form of the gradient 
of the function is to assume the form of the function itself or of 
its derivative. These choices are more obvious, but less fruitful. 
In the first case, integration of V to find Vis prohibitive, and in 
the other, it is difficult to make the assumed form of V general 
enough to give the required flexibility. 

The stability theory can be applied in several ways. A par¬ 
ticular-system can be studied, or conditions on the system and 
its initial conditions for stability can be obtained. Not only can 
stability be determined, but also upper and lower bounds on 
the magnitude of the state vector can be obtained. 

When the Liapunov function is known for one system, it may 
be used for other systems which are not too different. This fact 
has been used in many previous treatments; Hahn 1 uses linear 
bounds on the auxiliary term,? (x) in the equation x = Ax + g(x) 
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with the Liapunov function for the linear equation x — Ax. 
Antosiewiez 8 bounds the norm of the additional term g in the 
equation x = f(x, t) + g (x, t ) by the inequality \\g\\ < h (t) Ml 
with h< 0, J 0 °° h (t) dt < oo. In the present paper linear bounds 
are used to assure the stability of the equation x =/(*) + g (x, t). 

This work is strongly motivated by the similar treatment in 
Hahn 1 and it is felt that the slight generalizations are significant 
for the application of this theory to the design of control 
systems. Examples of the use of this theory are given for second 
and third-order systems with a linear principal term. 

Discussion 

When a controller is designed for a system there is usually 
some uncertainty about the exact mathematical description of 
the forces which will act on the system. For example, in the 
design of an autopilot for an aircraft, there is an uncertainty of 
the exact functional relation between the angle of attack, the 
control surface deflection and the torques on the aircraft. The 
design is usually carried out by using the best available estimate 
of this relation and considering the linear model obtained from 
the first term of the Taylor’s series expansion of the function. 
The design of the controller is then based upon this assumption 
and actually designed to operate for ‘reasonable’ changes from 
the model. It is desirable to have a controller which will control 
the system in a way which is insensitive to the actual torque 
relations that occur. This point of view leads to consideration 
of ‘adaptive’ controllers or the use of ‘invariance’ relations. 

Using the theory of Liapunov it is possible to obtain estimates 
of the class of functions for which a given controller will operate 
satisfactorily. Estimates of this type are the subject of the 
problem of Aizerman, and the controllers of Lurie and Letov 
also seek classes of functions for which satisfactory operation 
is obtained. 

The method for estimating the class of function in the prob¬ 
lem of Lurie is concerned with one scalar function of one 
variable; the rest of the system is linear. In the following, 
bounds are obtained on a time-varying vector function of the 
state of the system. In the discussion of this method in Hahn 1 , 
bounds are obtained on the components of the non-linear 
function, and it is required to remain between fixed linear forms. 
That is, 2 n constant vectors are chosen such that either 

x T a i <g i (x)<x T b i (i = 2,..., n) 

° r x T a i >g i (x)>x T b i 

with the vectors and bi chosen from a suitable set of vectors. 
This theory can be generalized somewhat to allow the use of 
a set of linear forms to bound the components. This generalization 
is important to the application of the theory because the func¬ 
tion g (jc) is thus not required to coincide with the intersection 
of the two hyperplanes. (The inequalities must be satisfied for 
all 3c. For those values of x for which x T = x T bi the require¬ 
ment is quite stringent.) Also the components g 4 can be functions 
of time as long as the linear bounds are satisfied for each instant. 

Mathematical Derivation 

The dynamics of the system are represented in the n dimen¬ 
sional state space by 

*=/(*) + £ CM) /(°)~ 0 ^ 

x(t 0 ) = x 0 g (0,0 = 0 


The ‘norm’ of the state vector ||x|| is given either by the Eu¬ 
clidian norm \\x\\ = (x x 2 + ... + xffi = (x T x)% or by some 
suitable norm such as ||x|| = (x T Bx)% with a positive definite 
B matrix or ||x|| = (V(x))± with V(x) positive definite. (This 
norm is not the usual norm of a linear space since the triangle 
inequality and the condition |MI = M Ml are not neces " 
sarily satisfied.) Assume a Liapunov function V (x) is known 
for the system y=f(y) (2) 

with the property that its derivative for eqn (2), F 2 = VL*/, is 
negative definite. Let X be a fixed open region containing the 
origin, let h be a fixed positive constant, and consider the set 
P h of all n x n constant matrices G for which 

VF •/(*) +VF Gx< -h \\x\\ 2 (3) 

for all x e X. 

Let V 0 be the largest region in state space which is bounded 
by F (x) = constant and contained in X. 

The matrix G a can be considered to be composed of the 
vectors gf* (/ = 1,2, ..., n) with 

giot — {gil> gil-> •• •» Sin)a 

5=5 [£la> glcv ** *9 ($) 

These vectors will be used to generate linear forms, x T g ia . 

A set of matrices R in G space will be called symmetric if the 
following condition is satisfied. If the zth row of one matrix in 
R (i = 1,..., n) is replaced by the ith row of another matrix 
in R then the resulting matrix is also in R. A symmetric subset 
of P h will be designated by R h . 

On the basis of these definitions the following theorem may 
be stated. 

Theorem of Linear Bounds- If the following conditions are 
satisfied 

( d) F 00 is a Liapunov function for eqn (2) with a negative 
definite derivative for y e Y; 

Qb ) For each fixed (x, t) in (X, t > t 0 ) 2 n vectors g ia and g ip 
can be chosen from some G a and G p in Rh such that the inequ- 

alltlCS x T g ix <gi (x, t)<x T gm (i=l,2.n) (6) 

or 

x T g ia >g i (x,t)>x T g ill 


are satisfied, then the equilibrium position of eqn (1) has the 
same stability behaviour as eqn (2) in the small. In the case of 
asymptotic stability, the region V 0 is included in the entrance 
region. 

Proof -—The inequalities of eqn (6) assure that V (a) is also 
a Liapunov function for the system of eqn (1). This can be 
shown by considering the derivative of V for eqn (1): 

F 1 = VF-/(x) + VF-gO,f) (7) 

By multiplying the ith inequality from eqn (6) by the ith com¬ 
ponent of VF, arranging the inequality with the greater term 
on the right, and adding the results for all /, 

VV-G 1 x<VV-g(x,t)<WV-G 2 x (8) 

where G x and <7 2 are made up of rows from G a and G fi . Due to 
the assumed symmetry Gj and G 2 are in R^. Then eqn (3) implies 

that Fi< —h ||x|| 2 for xeX 
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Remarks 

If h = 0 in eqn (3) and V (x) is a definite function, then the 
stability of eqn ( 1 ) can be concluded. 

The inequalities ( 6 ) which are sufficient for asymptotic 
stability or for instability are of course not necessary. It is easy 
to construct examples for which the system is stable even though 
the term g ( x , t) is not in the region defined by these inequalities. 
Nevertheless, the result is general enough to provide system 
design criteria for a class of non-linear systems. 

The region R h is obviously dependent upon the characteristics 
of the term/ (x). This is consistent with the fact that the term /(x) 
can be designed to ‘overpower’ the term g (.x, t). This dependence 
can be used to determine the type of control/(x) which accom¬ 
modates the greatest uncertainty or variation in g (x, t). 

The region R h also depends upon the Liapunov function V, 
and for fixed f (pc) it is desired to choose V in order to give the 
most suitable bounds on the function g (x , /). 

If only one component of g is non-zero, the inequalities 
( 6 ) can be interpreted geometrically in the n + 1 dimen¬ 
sional Euclidian space ( x , z). In this space there is a region, Z, 
made up of points which satisfy the inequalities 

x T g ia <z<x T g ;fi 

for some g ia and g t9 in If the function z — g t (x, t) is always 
inside the region Z, then the system is asymptotically stable 
(unstable). The boundary of the region Z is found by taking 

max [x T gJ (10) 

. 9i a 6 R h 

and 

min 
Qia £ Rh 

for each x. From the linearity of these relations it is clear that 
the vectors g ia corresponding to the boundary of Z are on the 
finite or infinite boundary of R h . The boundary of Z consists of 
rays from the origin and thus a map of this region may be made 
on the unit sphere. If the component g t (x, t) is only a function 

of one of the coordinates, x j i.e., . 4 /)=^ 

(0, 0, x j9 0, t) then the region Z can easily be drawn as in Figure 1. 



Figure 1. The region Z when the non-linearity is a function of only one 
variable , Xj 


The technique requires the use of a known Liapunov function. 
While there are a few methods of constructing these functions, 
such as the method of Zubov 1 and the variable gradient method 
due to Schultz 7 it may be more efficient to let/(x) = Ax + g 0 (x), 
where A is a constant matrix with the components 



dxj 


x = 0 
f = f 0 


( 11 ) 


The term g 0 is then combined with the term g (x, t). In this case 
one may choose 

V=x t Bx with B—B T (12) 

and 

A t B + BA=~C (13) 

with C chosen positive definite as usual, or one may choose any 
positive definite B. The region P h is determined by eqn (3) 
which now reads 

* T [(A + G) t B+B(A + G) + hI]x< 0 (14) 

Thus taking 

M=-[(A + G) t B + B(A + G)'] (15) 

The region P h is that region in G space for which the quadratic 
form j, 

x (M~hl)x> 0 x^O (16) 

is positive definite. If only one row of G is non-zero, P h is a 
symmetric region. 

If two positive constants K and k can be chosen such that 
K\\x\\ 2 >V(x)>k\\x \\ 2 (17) 

then the magnitude of the state vector is bounded by the inequality 



This inequality may be used to assure proper transient response 
of the controlled system. For the case ||x|| 2 = V, K = k = 1. 

When eqn ( 2 ) is asymptotically stable in Y and eqn (3) can 
be satisfied on the exterior of some region F cz Y, then eqn (6) 
implies that the state of the system ( 1 ) asymptotically approaches 
the region bounded by the minimum surface V = constant 
which contains the region F. 

Application of the Technique of Linear Bounds to a Second-order 
System 

As an example of how this theorem may be applied in the 
design of control systems, consider the simplified rotational 
dynamics of an aircraft about a single axis. The system to be 
considered is shown in Figure 2 . 

The dynamics of this system can be written in the form 

6 —co 

cb =f x ( 6 , co) -f- K D f 2 (0 , u) (19) 

u = —(9 — 6 0 )—K r co 

The control input u may be eliminated to obtain the second- 
order system 

9 — co 

cb =/i (0, m) + K D f 2 [0, -(0- 0 O )- K r co] ' 



Figure 2. Simplified dynamics of aircraft control about a single axis 
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If an equilibrium position exists then the equation 

/ 1 (0 e ,O)+X B / 2 (0 e ,0 o -0 e ) = O (21) 

must have a solution at 0 = 6 e . This is the requirement that a 
trim point exists. If there is a trim point then the variables 

x 1 = 6 — 9 e 

x 2 = cd (22) 

(0 e = constant) 

give the deviations of the system from the trim point, and the 
dynamics become 

i 1 = x 2 £23) 

X 2 =/l ( x 1 + ^ X 2 ) + Kd/i ( X 1 + ^ $0 “ X 1 8e “ &r X l) 

By choosing the numbers a x and a 2 such that (— a x x 2 — a 2 * 2 ) 
forms a reasonable linear approximation to the function 

/i+U 

and taking 

g 2 (^ij * 2 > flo) = /i + Knfz + a i x i Jr a 2 x 2 
the dynamics now read the same as eqn (1) with 


The coefficients a x and a 2 are chosen to be constant even though 
the partial derivatives may vary. The variation is part of the 
term g 2 (x l5 x 2 , 0 O ). For this example it is assumed that a x = a 2 == 1. 
Then the matrix B is given by 


(24) 



0 1 

x x 


0 

(x) = ^4x = 

^ 1 ^ 2 _ 

_ x> 2 ^ 

, g = 

_g2( X U X 2’ 9 o)_ 


(25) 


The Liapunov function for the equation 

x = Ax (26) 

is constructed as in the above remark. Taking the matrix C I 

in eqn (13) the elements of B are obtained by solving the equations 

0 0 2a x , , A*. , , 

0 -2a 2 2 I UJ = |-l| (27) 

1 —a t —a 2 

The solution is 



bn 


-i 


^22 

= 

-1 


b i 2 - 


0 . 


?ir 


i>22~ 


b i2~ 


a 2 a x 

1 


2a 1 a 2 + '2a 2 


2a x 


1 . a i 

2 2a 2 

1 


B-- 


(30) 


1-5 0-5 
[0-5 1-0. 

For this Liapunov function the eigenvalues are approximately 
K = 1-81 and k = 0-69. 

The vector g (x, t) has only one non-zero component and 
the matrix G may be taken to be 


G = 


10 0 

L&21 S 22 J 

Thus, the matrix M has the form 

l-g2i — g21 - 0-5g 22 

—g 2 i _ 0-5 g 22 1 —2g 2 2 

for x = x T x and the matrix M — 1/2 / has the form 

T 


(28) 


— a. 


dx x 


+Kd \^1 dx : 


Sfl rs rs S/z 


~ dx 


~—K d K r 


dx q 


M — 


■] 


(31) 


(32) 


M- 


I = 


— gzi 


-g 2 i-0-5g 22 


— g 2 i - 0-5g 22 ^ 2 §22 


(33) 


The requirement that M- 1/2 / is positive definite gives 
the inequalities 

1 

-x— g 2 i>0 


2 g 22 > 0 


(34) 


g J i 2 ~ + 


g22 + 


g21 


2 ) '4 

The first two inequalities are satisfied in the region 

1 


g21< 

g22< 


(35) 


the matrix B must be positive definite for stability of the linear 
system and this condition is satisfied for % and a 2 positive. From 
the bound on the state vector in eqn (18) it is desirable to have 
the eigenvalues of B as small as possible for rapid transient 
response. The requirement that the matrix (yl — B) be 
positive definite, assures that K< y. This requirement places 
restrictions on the elements of the A matrix, and as a result the 
acceptable values of K D and Kr may be obtained. The coefficients 
a x and a 2 are approximately given by 


The third inequality is a quadratic form. This equation can 
be simplified by a rotation of axes defined by 

1 r 1 -°- 5 i h (36) 

_t> J (1-25)*L+ 0 ' 5 1 J L 22 J 


(1-25) 

then the quadratic form is 


u 2 < - 


V 


1-118 


+0-2 


(37) 


(29) 


The region R x i 2 shown in Figure 3 is bounded by this parabola. 
By similar computation the region R 0 is obtained. Region Z x i 2 
is shown in Figure 4. 

The region R 0 is, as mentioned previously, only sufficient 
for asymptotic stability and, since the solution to the Aizerman 
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Using the variable gradient technique 7 on the equation 


x 2 = ~~m 1 (x 1 )~m 2 (x 2 ) 
one easily obtains the Liapunov function 


with the derivative 


Jo + 


V=-2 x 2 m 2 (x 2 ) 
giving the conditions 

x i m i (*i)>0 x x =£0 

X 2 m 2 (x 2 )>0 x 2 ^0 


lim m i (O d£ = oo 

xi~*oo J o 

which are sufficient for asymptotic stability. If the function 
&2 (*i» x s) is the sum of two functions so that 

&2 (. x h x 2 > ) == ^1 (^1) — ^2(^2) ( 43 ) 

then one obtains the conditions 



Figure 4. The region Z corresponding to R 1/2 for the second-order system 


problem for n = 2 has been obtained, it is known that 
fai - Ote 22 = 0) a sufficient condition for asymptotic stabii 
“ gn< 1 tea < 0- The treatment by the method of lin 
bounds is more general since the non-linear terms may depe 
upon more than one component of the state vector. Howev 
this generality may lead to results which are too restrictive. 
It should be noted however that this set of linear bounds 

much less restrictive than the bound 


Usll^ c oM (38) 

smce c 0 < 1 and this corresponds to a circle of radius 1 in the 
£zi £22 plane. 


[aiXi + n^xjJx^O (<0) x t >0 (<0) 

[a 2 x 2 + n 2 (x 2 )]x 2 >0 (<0) x 2 >0 (<0) ' 44 ’ 

These relations correspond to an R 0 consisting of the entire 

quadrant g 2x < u v g 22 < a 2 for the form of non-linearity in 
eqn (43). 

However, because h = 0 for this Liapunov function, the 
bound on the transient response given by eqn (18) is trivial. 
A bound on the response is important for control system 
applications. 


Application to a Third-order System 

In addition to the dynamics of the airframe, the dynamics 
of the control actuator may be important. Consider the system 
m Figure 5. The dynamic equations are 

d = co 

cb=a*(6,u) ( 45 ) 

d = —d j (0— 9 0 ) — d 2 (o—d 3 u 
The equilibrium position is given by the solution to 
a *( 0 e > 0 = 0 

-<*i(0.-0o)-d 3 «.-O (46> 



Figure 5. Third-order system in which the torque is a function a* of the 
attitude and control deflection 
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or, combining these equations, 

0 -A 


(o.-e 0 ) 


=o 


(47) 


If at each instant of time this equation has the solution 


then the variables 


0 = 0, 
u = u e 

(48) 

x t = 0—0 e 


x 2 =co-d l . 

(49) 

x 3 = u — u e 



define the deviations from the equilibrium position. 

The deviation dynamics are given by 

Xl = X 2 

x 2 = a* (x t + 0 e , x 3 + u e ) - 0 e (50) 

X 3 = — d ^X j ^2*^2 d 3 X 3 ^e 

By assuming that the equilibrium position is fixed and making 
a linear approximation to the function a*, one obtains 


*i 


■*2 

= 

*3- 




*1 


0 


x 2 

+ 

&2 (^ 1 > -^3) 

_ 

* 3 . 


0 


(51) 


0 1 0 

-fc, 0 fc 3 , 

-d 1 ~~d 2 —^3J 

There arc six equations from eqn (13) to solve for B. Taking 
k x - 7*15, k % = 3*58, d x - 575, <4 - 143, d z - 15, the matrix 5 
(with C =-= 1) is given by 

12-5 0-188 -0-00146 

0-188 14 0-00481 

-0-00146 0-00481 0-035 

The eigenvalues of the matrix A (the ‘poles’ of the linear 
part of the system) are 


B = 


(52) 


X x = —4*6 
^ 2 , 3 = — 5-1 ±J 21*1 


(53) 


and the eigenvalues of B are 


The bound given by eqn (38) is quite similar to that cor¬ 
responding to the region obtained above, and the bound on the 
transient response is too conservative to be useful. However, 
by using the actual minimum of (— V/V) one may improve 
this estimate. 

An alternative way of selecting the B matrix is as follows: 

B = [$5 7 ']“ 1 (58) 

where 0 is the matrix of eigenvectors of the matrix A. In the 
normal coordinate system 

z = $~ 1 x, $~ 1 A$ = A (59) 

the dynamics of the system are given by 


^1 — ^i z i 


(60) 


A reasonable norm for the state of the system is ||z|| =(Fz)i 
This norm is suitable for use as a Liapunov function and has 
the property that 


y< 2 R e X m < 0 (for stability) 


(61) 


where X m is the eigenvalue with the maximum real part. 

This same norm can be used in the x system of coordinates 
and is given by n z f =x r Bx (62) 


with B from eqn (58). For the third-order system above, this 
norm satisfies the following inequality 


NI<e~ 4 MM (63) 

It is suggested that this norm is more useful than the rather 
arbitrary norm x T x. For g ( x , t) in R lh defined using this norm, 
the inequality eqn (18) becomes 


||z||<e 2 ||z 0 || ( 64 ) 

The matrix B given by eqn (58) for the system of eqn (51) 


B= 


2-138 0-273 0-015 

0.273 0.0617 0.00416 
0-015 0-00416 0-00151 


(65) 


and the region R\! 2 is also shown in Figure 6 for this case. 


12-55 

fc 0 = l-395 (54) 

lc= 0-0345 

The bound on the transient response in eqn (18) becomes 

t 

||x|| < 19-1 e |M (55) 

for the Euclidian norm x T x = ||x|| 2 . This bound is quite con¬ 
servative since from the solution of the linear system a more 
reasonable estimate is 

[|x|| <20e -4 ' ||x 0 || (56) 

The matrix G has only two non-zero components, g n and g 2 3 . 
The region i ? 0 f° r these parameters is shown in Figure 5 and is 
given by the elliptic region 

1 -959 g\ +0-0018 g lg2 +1-989 g \+0-376 gl +0-0096 g 2 < 1 ^ 


923 



-2 


Figure 6. The regions R 0 and J ? l/2 for the two different Liapunov func 
tions for the third-order system 
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Conclusion 

A theorem of linear bounds is proved. This theorem may 
be used to obtain a class of similar systems for which satisfactory 
response is assured. The class found by this method is not the 
whole class of satisfactorily operating systems but often gives 
a class large enough to be of practical value. The most suitable 
norm for a linear system with constant coefficients is the length 
of the state vector in normal coordinates. 

The research leading to this paper was supported in part by 
the United States Air Force under Contract No. AFOSR-62-68 
monitored by the Air Force Office of Scientific Research of the 
Air Research and Development Command. 
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DISCUSSION 


J. C. Gille, Ecole Nationale Superieure de VAeronautique, 32 Boulevard 
Victor , Paris , France 

The second method of Liapunov is very elegant, but the application is 
rather difficult, since there is no general method to find the suitable 
Liapunov function. 

Mr. Nesbit starts from the form of a standard which he introduces 
as a mathematical notion. It seems to me that it would be possible to 
introduce similar forms based on physical notions, particularly of the 
conservative energy in the sense of the Lagrange equations. I should 
like to give here the main ideas of the energy theory of stability that I 
suggested some time ago 1 . Let the non-linear equation be, for example 

ax +f(x 9 x,t)x + fix = 0 

The conservative energy of this equation (or, more particularly, the 
energy of a dynamic system described by this equation) will be, after 
the introduction of the new variables 

pr 1 =x v /«, w 2 =x Vis 

E = W? + W 2 = || W\\ 

W x and ) W 2 may be called the plan of rectangular coordinates, in this 
plan the distance from the origin being a measure of energy. The 
system is stable if this conservative energy is diminishing in a mono¬ 
tonic way, i.e. if 


this corresponding with Liapunov’s second condition. 
In the case of the Hahn equation, for instance, 


and 


and thus 


x + ax + bx 3 A-x = 0 


-r. 


E = x + xr = (a + bx 2 )x 2 dt+C 

' o 


(in case a < 0, b > 0 ). Based on the notion of the energy plan, 
we conclude that there is stability inside of the circle centred at the 
origin, the square of the radius being equal to—alb (Figure A). 
The notion of the energy plan, the conservative energy plan and 



of the conservative energy permits, as a consequence, to find 
not only the condition of unlimited stability, but also of the total 
stability within a certain domain as compared with the initial con¬ 
ditions x 0 , x Q 

xl+xl<-2j- 

We hope it will be possible to extend the energy theory of stability 
to systems of a higher order, but for this some new notions should be 
introduced, such as conservative and dissipative coefficients, conserv¬ 
ative energy and the associated conservative system. 

We also hope to present a more complete theory for the next 
I.F.A.C. Congress 

Reference 

1 Gille, J. C. and Wegrzyn, S. Associated conservative systems and 
non-linear stability. Automatisme VII(S) (1962) 
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which means that 


dJE 
d t 


= a + bx 2 <0 


x 2 <- 


a 

~b 


The use of energy and momentum integrals for Liapunov functions is 
consistent with the historical motivations of the theory. In active 
systems the possibility of making the system unstable (making the 
‘energy’ increase) gives stability its practical significance. I am inter¬ 
ested in the general problem of determining Liapunov functions and 
look forward to the extension of the energy method. 
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M. [VI an sour, Institut fur Automatik und Industrielle Elektronik ETH , 
Zurich, Switzerland 

In order to prove the theorem of linear bounds the author selected a 
subset R h from the set P h in G space for which the Sylvester inequali¬ 
ties are valid or, more generally, for which the derivative of Liapunov 
function are sign definite. It is not easy to select this subset, especially 
when the system is of high dimension or if the original system is 
already non-linear. Sometimes it is not possible to find a symmetric 
subset in the G space because it does not exist, e.g. a system where the 
equations are 

i f = -AiZi+ZO) i = h-,n 

This can be put in the vector form 2 — AZ + g(z) and the non¬ 
lineary /(0) represents a non-linear element in series with a linear ele¬ 
ment in a closed loop. If wc succeed in selecting a symmetric subset, 
e.g. by taking all the rows of G equal to zero except one for the sake of 
simplifying the calculations, this will impose severe restrictions on the 
non-linearity, especially if the system is of high dimension so that the 
method may not yield practical results. 


R. A. Nesbit, in reply 


A symmetric set exists since G — 0 is one element of the set. The 
following procedure can be used computationally to determine a 
symmetric set of matrices: Choose ‘directions’ in G space by specifying 
possible rows for the matrices G v > Form a setL of matrices by choosing 
each combination of the specified rows. Multiply each member of the 
set L by a scalar, k, r The derivative ^ is now a function of the scalar 
k<i, and is negative definite for ki = 0. The interval — < k t < 

for which Kj remains negative definite must be determined from 
eqn (3). From the intersection of all the intervals one can obtain an 
interval — a < k < b. 

The set obtained by multiplying each member of L by any k : 
— a < k <! b is a symmetric subset R n . This method is clearly not 
practical when the number of possible rows is so great that the com¬ 
binatorial number of elements in L makes computational time pro¬ 
hibitive. Furthermore the results will be disappointing if the interval 
[— a , b] is very short. In the example 


f.r= — i-{-f(&) (i — 1,2... n) 

n 

<r= X <*i z i 

i = 1 

The choice of a symmetric subset is especially easy since all the rows 
of G can be taken the same. Further simplification results in this case 
because the argument of the non-linearity is a linear combination of 
state variables and thus only one direction in G space is of interest. 
Instead of a combinational analysis, only one matrix G need be con¬ 
sidered: r r „ "I 

Ct X 0C 2 ...°Cn\ 

G= a t oc 2 ...cc n 
_«i oc 2 ... 

From this one matrix, one obtains bounds of theioim-oiO<f(a)<ba. 
It is important to recognize that it is not necessary to take all rows 
equal to zero, except one, to form a symmetric set of matrices. 


number of equations to aid the choosing of Liapunov function V. 
This can be carried further to higher order cross-derivatives, and say 

that 0 3 F _ 8 3 K _ 0 3 F 

dxidxjdx k dxjdxj-dxi dx^x^xj 

This gives 

n(n — 1) — 2 

2 

more equations to aid the choosing of the Liapunov functions. 
However neat the method may be, solving several partial differen¬ 
tial equations simultaneously will be no easy task. It is interest¬ 
ing to note, that nearly all systematized methods of constructing 
Liapunov functions come back to solving partial differential equations 
such as Zubov’s method, Schultz’s method and Mason’s extension of 
Zubov’s method. 

P. C. Parks, Department of Aeronautics, The University, Southampton, 
England 

The author is concerned with aeronautical stability problems. I was 
concerned with the solution, by the second method of Liapunov, of an 
interesting missile stability problem 1 involving a system of linear 
equations with periodic coefficients. With a quadratic form as Liapu¬ 
nov function, it was possible to obtain a useful sufficient stability 
criterion and, within the stability region so found, estimates of the 
decay of transient motion along the lines of eqn (18), due originally 
to N. G. Chetaev. 

Reference 

1 Parks, P. C. Pitch-yaw stability of a missile oscillating in roll via 
the second method of Liapunov. J. Aerospace Sci. 29, No. 7 (1962) 
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I thank Mr. Parks for giving an interesting application of the method 
to an equation with time-varying coefficients. 

G. P. Szego, Research Institute for Advanced Studies ( R.I.A.S .), 
Baltimore 12, Maryland, U.S.A. 

I congratulate Dr. Nesbit on his very fine paper, and make the follow¬ 
ing comment. The ultimate aim of the procedure presented is of course 
to find the best quadratic form which will yield the maximum allow¬ 
able range of variation of the non-linear perturbation g(x,t). In the 
case in which the non-linear perturbation depends only upon one of 
the components of the vector x 9 say x t , the characterization of the 
‘best quadratic form’ is quite simple: it is the quadratic form which 
allows the widest possible wedge in which *(*<) has to lie. However, 
when g(x,t) depends upon more than one component of the state 
vector x, the definition of ‘best quadratic form’ does not seem to be a 
trivial problem and depends upon the particular system under in¬ 
What particular figure of merit for the definition of best quadratic 
form’ did the author use in this work? Would he care to comment on 
this problem in general? 


M. R. Patel, Cambridge University, Cambridge, England 

I would like to present a small extension to Schultz’s method of 
constructing Liapunov functions via V V. The requirement for V to e 
single valued is that VxVF= 0. Another way of putting the same 
thing is that the second-order cross derivatives of V be equal, i.e. 

d 2 V = 6 2 F 

0 xfixj dXjdxi 9 

That gives us n(n — 1) 

2 


A. Nesbit, in reply 

is important problem of optimizing the choice of Liapunov function 
lot directly considered in the paper. As with all problems of opti- 
zation, choice of criterion is antecedent to computational solution, 
e comparative definition of value used by the author was that the 
ion of two ‘wedges’ is better than one ‘wedge’. 

Value is subjective, and the definition of ‘best’ Liapunov function 
ist depend upon the individual situation. However, m the case that 
> relative magnitudes of the bounds on various components of g can 
specified, there is still no convenient method for determining the 
Dtimum’ Liapunov function. 
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On the Estimation of the Decaying Time 

L. HWANG 


Summary 

In this paper Liapunov’s second method is applied to the stability 
for certain functions taking on given values, and some formulae, to 
estimate the decaying time in the general case, are obtained. The 
linear system with quasi-constant coefficients, and the quasi-reducible 
system, are studied in detail. 


Sommaire 

La seconde methode de Liapunov est appliquee a l’etude de la stability 
de certaines functions avec valeurs donnees. On a obtenu des formules 
pour 1 estimation du temps d’amortissement dans le cas general. Le 
systeme lineaire a coefficients quasi-constants et le systeme quasi- 
rdductible sont etudies en details. 


Zusammenfassung 

Dieser Aufsatz behandelt mittels der 2. Methode von Ljapunov das 
Stabilitatsproblem fur bestimmte Funktionen, die gegebene Werte an- 
nehmen. Daraus ergeben sich allgemeine Beziehungen zur Abschatzung 
der Abklingzeit. AuBerdem werden lineare Systeme mit quasi-konstan- 
ten Koeffizienten und quasi-reduzierbare Systeme im einzelnen unter- 
sucht. 


The problem of stability is one of the basic problems in the 
proper operation of any dynamic system. After Liapunov’s bril¬ 
liant work 1 , very great effort has been expended on its theory in 
the last twenty years 2 " 8 . In the monograph by Zubov 4 , the sta¬ 
bility of the invariant sets of the abstract dynamic systems in the 
metric space are treated in a very general sense. 

Consider the differential system 

^-s C*T? * * X m 0 (*S= 1, 2, . Tl) (1) 

It is often necessary to study the stability problem for the given 
values (j>i , ..., (/)% of the coordinate functions 


= (£ = 1,2, ...,X) 


( 2 ) 


Without loss of generality, = 0, ..., = 0 may be taken. 

Thus, the standard working state of the system is given by the 
equation 

Mx u ...,x„)=0 (i=l,2,...,X) (3) 


Let the set of points defined by (3) constitute an (n - k) 
manifold ^ (n — k~). By means of (3), some particular and inter¬ 
esting motions of system (1) can generally be described (for 
example, the self-excited oscillations or the motions which 
demand their characteristic functions to take on given values). 
Here the generalized stability differs from the Liapunov sta¬ 
bility in that the unperturbed motion is no longer a particular 
motion (e.g. trivial solution) but its K coordinate functions take 
on given values. In general, (3) represents a class of motions and 
constitutes a manifold in the phase-space. From the research 
point of view, the coordinate functions (2) are of more interest 
than the coordinates x lt x n themselves. 


Obviously, when </>* = X{ , i = 1, k, k < n, the stability 
of the partial coordinates is obtained, and when <p { = x { , 1 = 1, 
...,k,k = n, it agrees with the stability in Liapunov’s sense! 

In papers by Liapunov 1 and Rumyantsev 8 the stability of the 
partial coodinates and the stability for the given values of the 
functions are discussed. They require the absolute values of the 
initial perturbations x®,..., x® to be sufficiently small and thus 
essentially the unperturbed motion was supposed to be a point 
m the phase space. The approach in this paper differs from theirs, 
and it will be explained clearly below. 

The set of points which satisfy the inequality 

K 

^ ( l>f(x 1 ,...,x n )^H 2 (4) 

i=l 

is^ called the iT-neighbourhood ( n — k) and is written as 

^(n- k) (H). It is assumed tacitly that through each point of 
^ (n — k) ( H ) there exists a unique solution of the system (1). 

Of course, it is necessary that the standard working state (3) 
of the system has a certain upholding ability, which means that 
the functions 

= grad (i = 1,..., K) (5) 

should satisfy the conditions 


• • * 5 x ni t) = 0 as (n — k) (6) 

or equivalently, 3F {n - k) is an invariant set of the system (1), 
where {x} n represents the /2-dimensional vector. 

Definition 1. The system (1) is said to be stable with respect 
to the functions (2), taking on zeros (3) whenever, given any 
e > 0, there is a d (t 0 •£)>(), such that, for all trajectories 
a: (t) with initial values satisfying 


{x(to)}n = {x°} n e&(n-k)(d), 

one has 


for all t > t 0 . 




fo >0 


(7) 

( 8 ) 


Definition 2. The system (1) is said to be asymptotically 
stable with respect to the functions (2) taking on zeros (3) if 

(a) Definition (1) holds, 

(b) , * 

lim 4 r=lim £<£?[>,(f),...,x„(0]=0 (9 ) 

t~* oo t~* oo i = 1 

i.e. for any given rj > 0 there is a positive number T=T(w u x°) 
such that h 09 

{x ( t)} n eS? (n- k) 0 rj ) as t>t 0 + T (10) 

Definition 3. The system (1) is said to be equi-asymptotically 
stable with respect to the functions (2), taking on zeros (3) if 
(a) Definition 2 holds, 
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(b) there exists H > 0, and T^T(rj), such that for all 
trajectories {x (r)} w with initial values satisfying 


(b) V satisfies Definition 7, 

(c) the total derivative 


t 0 > 0 

sre 

{*0o)}„eG 0 t o >0 


(ID 

then (10) holds. 

If the initial conditions were subjected to the following 
restrictions 


( 12 ) 


then the above-mentioned stability, asymptotic stability and 
equi-asymptotic stability are said to be stable, asymptotically 
stable and cqui-asymptotically stable under condition (12) 
respectively. 

In the sequel, the function V (x l9 ...,/) is called the Liapu¬ 
nov function with respect to functions cj) 1} if 


V ( x x , t) = 0 as {x} n e^(n-lc) (13) 

and V is assumed to have continuous partial derivations. 

Definition 4. The function V (x l9 ..., x ny t ) is said to be 
positive (negative) semi-definite with respect to (2) if 

(a) (13) holds, 

(b) V> 0 [F< 0] inJF (n - k) (H). 

Definition 5, The function V (x l5 .. x n , t) is said to be positive 
(negative) definite with respect to (2) if 
(#) Definition 4 holds, 

(/>) there is a positive function W 1 (y l9 . . y K ) such that, in 
JF (« - k) (H\ 

V (x[,.. x„ t)> W x l<Pt (Xj.X„), ...,<t> K (x u ...,x„)] (14) 

{V (x u -,*»)> -,<Pk(xu -,*»)]} 

Definition 6 . The function V is said to be uniformly small, 
if for any given e > 0, there is <5 (t) > 0 such that the conditions 
/ > 0 and {x} n eSF (n — k) id) imply V < e. 

Definition 7. The function V(x l9 t) is said to have 

infinitely small upper bound with respect to (2) if there is a 
continuous function W 2 *• •> T/c) such 


dV 

d t i-i 


=^+gradF-X 


(18) 


is negative definite with respect to (2), then the system (1) 
satisfies Definition 3. 

(B) If the system (1) satisfies Definition 3, and the rank of 
matrix 


DQftl? 0 k) 
D Oq,...,x„) 


Q> 

■e- 

H - 1 

0^1 

fix,. ’’ 

ox, 

dcj) K 


Mi’' 

0X, 


(19) 


is K, and the functions (i = 1, x) defined by (5) are 
uniformly bounded in J r (/i — k) (H), then there is a function V 
which satisfies all conditions in {A). The proof of this theorem 
is given in the Appendix. It is not difficult to prove the following 
corollaries. 

Corollary 1. If Fsatisfies Definitions 5 and 6, and dV/dt | o) 
is negative semi-definite with respect to (2), then the system (1) 
satisfies Definition 1. 

Corollary 2. If F satisfies Definition 8, and dV/dt | o> is 
negative definite with respect to (2) then (9) holds for any 
t 0 > 0 and any x° in the space. 

Let the set of position points at time of the motions which 
take on the initial positions in G° be written as G (0 . 

Corollary 3. If F satisfies (A), Corollary 1 or Corollary 2 
in Gton&in- k)(H), then the system (1) is stable, equi- 
asymptotically stable, or asymptotically stable in the whole 
under condition (12) respectively. 

Corollary 4. If the system (1) satisfies Definition 3 under 
condition (12) and the rank of matrix (19) is K in the neigh¬ 
bourhood S?(n-k)n G® and (i = 1,.. k) are uniformly 
bounded in G® n (n — k ) (fiT), then there is a function V 
which satisfies the conditions in Corollary 3. 


(a) W 2 (0t, ..., 0)-0, 

(b) in 3? (n — k) (H), 

W 2 \_<j) i (x u ...,*„), U--'9 X n> 0 (^) 

Definitions . The function V (x l9 ..., x n9 1) is said to have 
the property A, if there are two positive continuous functions 
W x (S) and W 2 (S) such that 

(a) H / i(0) = M / „(0)=0, Tf 1 (oo)=)f fl (oo)=+co (16) 

(b) W 2 (\m K )^V(x 1 ,...,x n ,t)^W 1 (\\<l>\\K) ( 17 ) 

Parallel to Definitons 1 and 3, one has the fundamental 
theorems shown in the following section. 

The Fundamental Theorems 

(A) For the system (1), if there is a Liapunov function 
V (x x ,.. x n , t) such that 

(a) V satisfies Definiton 5 and it is positive definite with re¬ 
spect to (2), 


Example —Consider the system 

x= ay—cx(bx 2 + ay 2 ) sin 2 * —5 

bx +ay ( 20 ) 

y= — bx—cy (bx 2 + ay 2 ) sin £p +fl -p 
(a‘b m c> 0) 

Obviously, if one takes <j> = bx* + ay 2 then (j) = \\kn 
(k = 1, 2,...) are the invariant sets of (20), they are closed 
orbits. By means of the Liapunov functions V = i (0 — 1/^0 
with respect to <j> - 1 \kn, the following statements can be 

proved: 

(a) In the exterior of the ellipse (j) == 1M there is no closed 
orbit; 

(b) in the interior of the ellipse <fi = l/#> there are infinitely 
many closed orbits; 

(c) the closed orbit is asymptotically stable when K is even 
and it is unstable when K is odd; 
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(d) the origin x = y = 0 is a singular point of ( 20 ) and it is 
stable. In any of its neighbourhood, there are infinitely many 
closed orbits, and hence the origin is not asymptotically stable 

In the control or the dynamic systems, it is often necessary 
to estimate the decaying time of perturbations for the standard 
working state. In this paper the problem of estimating the 
decaying time is considered. In the sequel it is assumed that 
system (3) is equi-asymptotically stable with respect to ( 2 ), and 
the following discussions are valid in certain attractive 
regions of JF (« — k). 

Let V be a Liapunov function of (1) which satisfies the 
conditions of the fundamental theorem (W). In the general case, 
there are two positive definite functions W 1 (y l9 y K ) and 
W 2 (>i> • * •> 7 k ) such that 


O)] >V(x u 

>W 1 l</) 1 (x), (x)] (21) 


Besides, it is assumed that there are two functions /* ( 5 ) 
an df 2 0 s ), such that in # (n - k) (. H ) the inequalities 


- , \ du. 


^/aO) 


hold. Furthermore, from ( 21 ) one has, in general, 

f{*„} e{W 2 <V 0 } implies {x}„ e{V<V 0 } 

1 K) e { v ^ e} implies {x}„e{PF 1 < e} 
where V 0 and s are given position numbers. Denote 


( 22 ) 


( 23 ) 


T,= 


r° di 
. fiW 


t 2 = 


p° d;. 

a MX) 


( 24 ) 


Then, the following theorem estimates the decaying time. 

Theorem 1. The decaying time T of the motion of the system 
( 1 ) from an initial point in the region 


W 2l4>i(x),...,<t> K (xy]<V 0 ( 25 ) 

to a point in the region 


w i ( 26 ) 

satisfies the inequality 

T<T 2 ( 27 ) 

The decaying time T from an initial point in the region 


W 2 l<l>i(x),..., 0 K (xy]>V o ( 28 ) 


to a point in the region (26) satisfies 

T>T X (29) 

Let Afg (I?) be the maximum value of W 2 on the boundary 
|| $ || K = R of # (n - k) (R) 

and let m x (y) be the minimum value on the boundary \\<fi\\ K =y 
of # (n — k ) (y). Again denoting 




'M 2 (R) 


d 2 


/ mi (y) fl (^) 

the following theorem is obtained. 


t 2 = 


'M 2 (R) 
ml (y) 


dA 

7 a W 


( 30 ) 


Theorem 2. The decaying time T of the motion of the system 
( 1 ) from an initial point in the region#' (n — k) (.R ) to a point 
in the region#' (n — k) (y) satisfies (27), and the decaying time T 
of the motion of the system ( 1 ) from an initial point in the region 
|| (j) || > R to a point in the region (n — k) (y) satisfies (29), 
where T v T 2 are defined by (30). 

By taking 

fi(v)=-xv f 2 (v)=- pv (a>fl) ( 31 ) 

one has 


T,= 



MAR) 

m 1 (r) 



M 2 (R) 
m x (r) 


( 32 ) 


Particularly, when fa = x h i = 1, ..., k < n one obtains the 
formulae to estimate the decaying time for partial coordinates, 
and when = x i9 i = 1 , n, then one obtains the formulae 
to estimate the decaying time for total coordinates ( 9 — 11 ). 

The above method is used to solve the following example. 

Example —Consider an autonomous system 


x = x-a 2 x 3 -b 2 xy 2 
y = y — a 2 yx 2 — b 2 y 2 ( <3> ) 

and its unique closed orbit 


( 33 ) 


([> = a 2 x 2 -f b 2 y 2 — 1 = 0 ( 34 ) 

If one selects the Liapunov function with respect to (f) to be 


V=(a 2 x 2 + b 2 y 2 - 1) 2 (35) 

then it may be asserted that: 

(a) the system is asymptotically stable with respect to <j> = 0; 

(b) the decaying time T of the motion of the system from an 
initial point in the region | <fi \ < 4> 0 to a point in the region 
\<l>\< e satisfies T < a 2 log (1 + e) fa/(l + fa) S ; 

( c ) decaying time T of the motion from an initial point in 
the region \ <j) \ < fa to a point in the region | r/> | < s 
satisfies T> b 2 log (1 + s) fa/(l + fa) e. 


On the Estimation of Decaying Time for a Linear System with 
Quasi-constant Coefficients 

In the study of a practical dynamic system, one usually takes 
the linear system with constant coefficients as its first approxima¬ 
tion. In general, the frequency method may be applied to 
estimate the time of transient process for the control system 
with constant coefficients. However, this method is only appli¬ 
cable to the case of single output under specific initial conditions. 
In addition, the method is not rigorous. This paper gives the 
formulae to estimate the decaying time in the general case, and 
the method is rigorous. 

A large amount of work 9-12 is devoted to the estimation 
of decaying time for the asymptotically stable system 

*s=Psi*i+ ... +Psn*n> S=1,...,N ( 36 ) 

where the coefficients pij are constants. There results may be 
summarized as the following. For any given positive definite 
quadratic form 

U=x'Ux 
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there is a positive definite quadratic form 
V = x'Vx 

such that 


V=x'Vx 

(38) 

AV\ 

, =-u 

Q-t ( 36 ) 

(39) 


If Mi and m 1 are, respectively, the maximum and the minimum 
eigenvalues of the matrix V, and M and m are the maximum and 
the minimum eigenvalues of the matrix U , then the following 
results are obtained. 

Theorem 3. The decaying time T of the motion of the system 
(36) from an initial point in 

EV=R 2 

5=1 


to a point in the region 


N 

I 4<r : 

S= 1 


(45) 

where u = max (oc i9 fa), v = min (oc i9 pj). It is easy to select the 
initial points such that the equalities in (45) hold (i.e. this 
estimation is accurate). 

In the following, the general formulae to estimate the 
decaying time is given. 

All roots of the characteristic equation D (A) = det (R — XI) 
= 0 are assumed to have negative real parts. Let X i9 ..., X l be 
negative real roots, written as X f = — (/ = 1,..., /), and let 

X l+l9 ... 9 X n be the remaining roots, written as @ 8 ±(o s i 
(S = 1,..., n — 7/2 = k) } and the order of the corresponding 
elementary divisors be n x ,..., n K . 

It is known that there is a non-singular linear transformation 
y == Cx to reduce the system (36) to the normal form (42), 
in which 

/M 1 0\ 


satisfies the inequalities 

^Lio g Mi^<T<^ilog^i$ (40) 

M 2 m 2 m x r 

In practice, it is of interest to select a suitable Liapunov 
function V, such that for the given system (36) the range defined ^ _ 
by (40) is as accurate as it can be. It is very difficult to answer the 1 
above question in the general case. But if the system (36) is 
normal and the elementary divisors of the coefficient matrix P 
are all simple, it may be proved that when 

N Tf om 

k- i *! 

5=1 

the equalities in (40) may be realized (i.e. the estimation is 
accurate). 

Let the normal transformation be 

y = Cx (41) , 

where C is a matrix with real coefficients, and the system (36) ^ ^ 

is reduced to the normal system 

y=Jy (42) ( ' 


1 0...0 

-a £ 1... 0 

\ \ 

X 1 

—0Ci 


-Pi-coi 1 0...0 

( 0 ,- fi , 0 1...0 




Tf one writes the x in ; matrix 

/I 1 


1+ 2^F 


this is constructed according to the following rule: 

(a) when s = a, is equal to ( l/oc) (1 + o), and let 

1 


«u ,) =: 


(b) when 


(43) 

J ~ <o 1 -P l 

\ 

-CO, 

0 a 

^ c0,-Pu 

V=x'C'Cx (44) 

may be taken as a Liapunov function of the system (36). By 
means of (42) the following results may be proved. 

Theorem 4. The decaying time T of the motion of the 
system (36), from an initial point in the (n - l)-dimensional 
ellipsoid V = F 0 to a point in the ellipsoid V = e, satisfies 




Thus the matrix is completely defined through the eigen¬ 
values - oci and the order of its elementary divisor. The maxi¬ 
mum eigenvalue of the matrix a( m i) is assumed to be v { 

(whenm,=l,Vj=— 


|Whenm i =2,v i =-+4^|_-+^4+ a 2 
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Following the method of construction of the matrix a( m i), 
the 2 fii x 2 rii matrix d 2ni may be constructed in the following 
manner 


(a) d^i- 1 ,2 j—ul ;-1 — 0, i, j = 1, .. ., 

(b) d^ 2 i -\, 2 j - i — 1,2 i-i =<2 a i) h j z 

(c) to replace by ft in the matrix 
For example — 2, one has 


i, ; = .,w f 


f (2) 

^11 

0 

a (2) 

a 12 

0 

1 


0 

a (2) 

a 12 

,(2) 

'21 

0 

a (2) 

a 22 

0 

l 

„(2) 

“21 

0 

a (2) 

a 22 


\0 

It is not difficult to prove that F satisfies 

£ 

af o.i v 


where »’ is the maximum eigenvalue of the matrix A, and it can 
be calculated by the aforesaid method. When m t = 1 or = 2 
it can be calculated through (49). If the maximum and the 
minimum eigenvalues of the symmetric matrix CAC are 
assumed to be M and m respectively, then the following theorem 
is obtained. 

Theorem 5. The decaying time T of the motion of the system 
(36) from an initial point in the (iV-l)-dimensional ellipsoid 
V = Vo to a point in the (IV-l)-dimensional ellipsoid V = e 
satisfies 

t 4 Io *T 

The decaying time T of the motion of the system (36) from 
an initial point in the sphere 


to a point in the sphere 


satisfies 


N 

Z *1=R 2 

s — 1 
N 

l^ = r 2 

S- 1 


is considered, where X is a vector function which contains the 
non-linear terms and the unknown components. If one constructs 
a Liapunov’s function (50) of its principal linear system 

x=px (54) 

and if one assumes X to satisfy the inequality 

|grad V'X\<bx'CCx , (b<2) (55) 

then the following results are obtained. 

Theorem 6. The decaying time T of the motion of the system 
(53) from an initial point in the sphere 


I xl = R 2 

S — 1 


Obviously, the formula for the maximum eigenvalue of d (2n $ is 
the same as that of in which oc t are replaced by ft. Consider 
the Liapunov function for system (36) to be 

V = x'CACx (50) 

where C is the normal transformation matrix, and 

/V rai) o \ 

I 1 


to a point in the sphere 


satisfies 


N 

Z *s=r 2 


V , MR 2 
T -2{l-bl2) l0g ^ r 


rrt - . MR 2 

r<viog— r 
mr 


Moreover, the system 

x=px+X(x,i) 


As an application of this theorem, an example of a forced 
oscillation is considered. 

Example —Consider the system 

u=pu+sU(u,u n )+F(t ) (57) 

where p is assumed to have all its eigenvalues with negative real 
parts, a is a small parameter, U is continuously differentiable 
and F (t) is the forcing term with period T. 

Let the system (56) have a periodic solution 

u s =u°(t), u°(t)=u° s (t+T) (s=l,..., N) (58) 

and let the linear transformation 

y = Cu (59) 

transform the system u = pu into its normal form 

y=Jy ( 60 ) 

By means of the transformation (59) the system (56) was reduced 
to a system 

y=Jy+eY(y) + $(t) ( 61 ) 

where 0 (0 = CF(t) has the same period as F(t ). Under this 
transformation, the periodic solution (58) is reduced to 

N 

= y° (t) = Z cos u° (0 ( 62 ) 

<T~ 1 

Consider the perturbations x s =y s - (t) then x satisfies 

x=Jx+eq(t)x+eX(x,t) ( 63 ) 

where q(t) is a periodic matrix with period T, and it may be 
evaluated through Y (t) and y° (t). If one takes f = C' 1 x, then 
£ u u (t) is the perturbation vector in u space. 

• ®y. means t ^ ie a bove method the matrix A is constructed, 
with its maximum eigenvalue v, and the maximum and 
minimum eigenvalues of the matrix C'AC are M and m respec¬ 
tively. The following results are obtained. 
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Theorem 7. The decaying time T of the motion of the system 
(57) from an initial point in the R neighbourhood of the periodic 
solution (58) to a point in the r neighbourhood of the periodic 
solution (58) satisfies 



where the term X in (62) satisfies 

igrad V’X\ <bx'x (b<2 i V=x'Ax) (65) 

and C is the maximum eigenvalue of the matrix q it) + [q (t)] 
when t e [0, T], By the above-mentioned r neighbourhood of 
the periodic solution (58), is meant the set of points which 
satisfies the inequality 

£ iu-u° s (t)f<r 2 

5 = 1 

On the Estimation of Decaying Time for a Quasi-reducible Linear 
System 

In the study of the dynamic systems, one may sometimes fail 
to approximate it by a linear system with constant coefficients. 
In this case one may take a reducible system as its approxima¬ 
tions, and construct the corresponding Liapunov function and 
estimate the decaying time. 

Consider the non-linear system 


satisfies 


^ v 1 MR 
T <”log—iv- 
2 6 mr 2 


Furthermore, if X satisfies the condition 

Igrad V-X\< bx'C'Cx (b < 2) (72) 

then one has the following results. 

Theorem 9. The decaying time T of the motion of the system 
(66) from an initial point in the sphere 


E 4=R 2 


to a point in the sphere 


satisfies 


N 

E * 2 =>- 2 

5=1 

v , MR 2 
^"2 (1-6/2) ° S mr 2 


(b< 2) 


where X satisfies (72). 

Since the linear system with periodic coefficients is a reducible 
system, and its characteristic number can be represented through 
its characteristic exponentials, the results in this section can be 
applied to the general periodic systems. 


Appendix 


x = p(t)x + X(x,t) (66) 

Let the linear approximation system 

x — p(t)x (67) 

be a reducible system. Assuming the characteristic number to 
be all positive, there is a Liapunov transformation 

y = C(t)x (68) 

which transforms the system (67) into its real normal form 

y=Jy (69) 

By means of the method mentioned in the previous section, 

V = x'C'ACx (70) 

is taken as a Liapunov function for the system (67). Since (68) 
is the Liapunov transformation when t > t Q , the maximum 
and minimum eigenvalues M and m cannot equal zero. Obviously, 
the maximum eigenvalue v of A can be calculated through the 
characteristic numbers of (67) by the same method. Parallel 
to Theorem 2 the following results may be obtained. 

Theorem 8. The decaying time T of the motion of the system 
(67) from an initial point in the sphere 

N 

5= 1 

to a point in the sphere 

N 

E *s =r2 

s= 1 


Proof of the Fundamental Theorem 

Obviously the system is stable with respect to (2) taking on 
zeros. 

For any given rj > 0 the region H > || (j>\\ K > disconsidered. 
From the conditions mentioned in the theorem, the function V 
takes on maximum M > 0 and minimum m>0 and the nega¬ 
tive definite function dV/dt | with respect to (2), takes on 
maximum — oc < 0. Let T = (M— m)/oc 4- 2/?, where ft is 
any arbitrary positive number. This is the required T and it 
is independent of the initial conditions. Thus, part (A) of the 
fundamental theorem holds. 

Parallel to Theorem 3, from the conditions of the fundamen¬ 
tal theorem, the following lemmas can be proved. 

Lemma 1. If the system (1) is equi-asymptotically stable with 
respect to (2), taking zeros for the initial values in ^(n - k) 
then there is i \t (r) such that the motions, defined by the initial 
points of the above-mentioned region, satisfy 

(a) ||0[*(£o + L x i> *o)]I1k^*K t ) 

(b) lim \j/(x)=0 f (t)<0 t>0 

T-*CO 

Lemma 2. For any given two positive functions M (rj) and 
i j, (ji), where M (rj) is an increasing function and lim ip (rj) - 0 
there is a function G (d) such that 

(a) G(rj)>0, G'(rj)>0 as rj>0 

(b) G(0) = G'(0) = 0 

00 f 00 

G[>(t)] dt < co G' (p (t)] M (Y) dr < co 

0 j 0 
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Lemma 5. If the system (1) is equi-asymptotically stable with 
respect to (2) taking on zeros, and the rank of the matrix (19) 
is K, then there are two positve constants A and X independent 
of t 0 and <fi° such that 

m< Ac , N 21 


where 


19*o 


W 


<Aq x 


4> 2 =T, 

s= 1 


4>t°0= are the initial values and t is replaced by 

*o "b Parallel to Theorem 3, the Liapunov function may 
be taken as 

V= 

then one has 
\dV 


E 4>s(t + r,<f> u .t) 

_s= 1 


dz 


Wc 


G' 


4> 2 (t + r,<j> u ...,<f> n ,t) 


d(f>" 

3 Ji 


dr 


<oo 


It is convergent and uniformly bounded. This implies that V 
satisfies the Lipschitz condition and thus V has an infinitely 
small upper bound. 




lt Li 


definite 

dV\ 


dFj; 

Hence 

dpj 


dt 11 


: G + ... + ] 


l-i 


is negative definite with respect to (2). 
The proof is complete. 
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DISCUSSION 


J. P. LaSalle, and G. Szego, Research Institute for Advanced Stud¬ 
ies, Baltimore 12 , Maryland , U.S.A. 

The theory developed in the first part of the paper has some aspects 
ich may be of practical importance in the investigation of the 
stability properties of invariant manifolds. There are, however, some 
questions concerning the meaning and significance of the stability 
concepts introduced and the validity of the ‘fundamental theorem’ 
when the manifolds are non-compact. 

-L® St l M 7 C trkH tS ° f th ® paper are relative the system of 

sSv ! ( ^ } defined by eqn(4) ' Because of ‘his, these 

y concepts can have, for example, the peculiar property that an 

mvanant manifold can be asymptotically stable in the sense of 

Liapunov and yet not be asymptotically stable in the sense of this 
paper. 

The difficulty is that the neighbourhoods &~{H )may contain points 
which are near infinity. The definitions of the paper are then state- 
ments not only about motions near the manifold, but also about 
motions that are far away from the manifold. The most obvious wav 
to avoid this difficulty is to restrict the definitionTto a Euclidian 

Stssurnffig ffiat f l th r manif0ld ' The diffiCUlty Ca “ also be avoided 
by assuming that limiting is positive as the distance from the mani¬ 


fold approaches infinity. We assume throughout the remainder of 
this discussion that one of these restrictions is in force and that 
the function is continuous. 

The next point we wish to make has to do with the ‘fundamental 
theorem. When the manifold is compact it is not difficult to show 
that the neighbourhood system of the paper is equivalent to the 
Euclidean neighbourhood system of the manifold. Thus for compact 
manifolds the stability concepts of the paper are equivalent to the 
usual Liapunov stability concepts for invariant manifolds. 

‘fundame e nTai e theo n r 0 em’ 0mPaCt manif ° ,dS * 8 diffiCU ' ty wHh thc 

All of this is not to say that the ‘fundamental theorem’ is not of 

wouffi litem 11 T y f Wdl h T applicati0 " s -1" ‘his regard we 

better m k lf e f mpl ? S have been worked ou ‘ which illustrate 
tter than tlle example of the paper the power of the theorem. 

a a° b f t0 know how the ‘heorem needs to be 

odified m order to be valid for non-compact manifolds. Here too 
it is relevant to ask: what is the relation between the stability concepts 
°f* e P a .P er and ‘he usual Liapunov stabilities relative to Euclidean 

zszsrxz state space? we wonder 
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ON THE ESTIMATION OF THE DECAYING TIME 


L. Hwang, in reply 

I thank Professor Lasalle for his comments and for pointing out the 
following: 

(1) In this paper stability is defined according to whether the 
norm \\(p\ \ approaches zero or not, rather than according to whether 
the ordinary Euclidean distance to the given manifold approaches 
zero or not. It is true that the phase point may have its limit point at 
infinity and thus the corresponding Euclidean distance may be in¬ 
creasing; it is not even difficult to give illustrative examples. However, 
since in practice we are usually interested in cp rather than the 
Euclidean distance, the latter concept is not introduced here. In fact, 
the norm defined in this paper is the distance in a more generalized 
space. 

(2) The stability definition and the method of treatment in this 
paper is a generalization of Liapunov’s definition and his second 
method. It is possible that a certain system is asymptotically stable in 
the sense of cp , as defined in this paper, while it is not asymptotically 
stable in Liapunov’s sense. The converse case is also possible. The 
definition of stability here is not equivalent to that of Liapunov. 

(3) The proof of the fundamental theorem does not demand the 
manifold itself to be compact. This theorem holds valid so long as the 
prescribed conditions are satisfied. The system given in Professor 
Lasalle’s comments x x — x\, # 2 = —*2 does satisfy the definition of 
asymptotic stability relative to cp = x 2 = 0 since x 2 = e 5 although 
the curve in the x x x 2 plane may not approach to the x x axis m the 
ordinary geometrical sense (Figure A). Moreover, for practical sys¬ 
tems it is usually assumed that the motion does not approach to 
infinity in a finite time interval. 

Finally, I would like very much to know Professor Lasalle’s works, 
if any, on the relation between the stability concept for invariant 
manifolds and the ordinary Liapunov’s stability concept. 


B. S. Razumichin, Institute of Mechanics , Academy of Sciences , 
Moscow , U.S.S.R. 

I have two questions to ask the author. 

(1) What is the meaning of the statement that all remaining co¬ 
ordinates are free ? ...... 

(2) What method is used to prove that the variety (complexity) 

under investigation is not empty ? 

I fully agree with Dr. Lasalle’s remarks. The definition given m 
the paper differs from Liapunov’s general definition only by the fact 
that the initial values of some quantities have no restrictions. This is 
obtained at an extremely high price, because it is absolutely necessary 
to prove that the variety under investigation is not empty, apart from 

stability investigations. ,, 

I cannot agree with the statement that the estimation 45 (Theorem4) 
is exact, as the inequality becomes equality only for two integral 
curves and at one fixed moment. On all other points of time axes the 

exact inequality is satisfied. .. , , , u 

I draw the author’s attention to my paper 1 which shows the 
estimate for separate coordinates obtained by Jacoby’s transformation. 

Reference 

1 p azitmtchin. B. S. Prikl. Mat. i Mekh. XXI, No. 1 (1957) 


L. Hwang, in reply 

(1) It is pre-assumed that cp ti = 0 (/ = 1,2,..., K ) is non-empty, which 
corresponds to the standard working state in practical application. 

(2) The inequality (45) also holds valid for a linear system with 
constant coefficients. In fact, when the system is reduced to its Jordan 
canonical form by means of linear transformation, there must exist 
two particular motions corresponding to the upper and the lower 
bounds of this inequality. 



Figure A 


New Methods for Constructing Liapunov 
Functions for Time-invariant Control Systems 

G.P. SZEGO 


Summary 

In the investigation of the stability properties of non-linear control 
systems one likes to have a general procedure for constructing Liapunov 
functions which will always yield a solution of the stability problem. 
The only general method in existence up to now was proposed by 
Zubov in 1955 and it is based upon the integration of certain quasi- 
linear partial differential equations. 

In this contribution the author generalizes and extends the work 
of Zubov, and proposes new types of partial differential equations 
which, as the Zubov’s partial differential equation, will provide a 
solution of the stability problem. On the other hand, these equations 
allow a certain freedom of choice of some parameters and some 
functions. The method developed, here will also give a nice intuitive 
interpretation of the meaning of Liapunov functions and of some 
problems of optimal control. 


Sommaxre 

Dans 1 etude des propridtes de stabilite des systemes de commande non- 
lineaires il est tres utile d’avoir une methode generate de construction 
des functions de Liapunov qui fournissent toujours une solution au 
probleme de stabilite. A l’heure actuelle la methode proposee par Zubov 
(1955) est la seule k remplir ces conditions; cette methode est basde sur 
integration de certaines equations quasi-lineaires aux derivees 
partielles. 

Dans la presente contribution l’auteur generalise et etend les 
travaux de Zubov et present de nouveaux types d’equations aux 
derivees partielles qui, comme celle de Zubov, resoudront le probleme 
de stabilite. Ces nouvelles equations laissent par ailleurs une certain 
liberte quant au choix de certains parametres et de certaines fonctions. 
Da methode proposee donne de plus une interpretation intuitive 
et elegante de la signification des fonctions de Liapounov et de certain 
problemes de commande optimale. 


Zusammenfassung 

Bei der Untersuchung der Stabilitatseigenschaften nichtlinearer Rege- 
lungssysteme mochte man ein allgemeines Verfahren zur Konstruktion 
Ljapunovscher Funktionen haben, welches immer eine Losung des 

^ a ^ ht f tSPr0b , emS llefert ' Die einzi § e bisher existierende allgemeine 
Methode wurde 1955 von Zubov vorgeschlagen, sie beruht auf der 
Integration gewisser quasilinearer partieller Differentialgleichungen 
In diesem Bettrag wird die Arbeit von Zubov verallgemeinert und 
erweitert und neue Typen von partiellen Differentialgleichungen vor- 

nmhlimf r ebe A nS ? W ‘ e die VOn Zubov eine L5sun § des Stabilitats- 

FreSeh n £ wtr rerSe n ^ ^ Gleichun S®" eine gewisse 
f/l Y h elniger Para meter und Funktionen. Die hier ent- 

Bedeuw der°r Slbt “I? ei " egUte anschauliche Interpretation der 

malen Relefung JaPUn0V U ” d V ° n ProbIemen der opti- 


Introduction 

In this paper some new techniques leading to a new approac] 

XS y of 


This study is limited to the investigation of the stability 
properties of completely defined control systems. The new 
method for stability investigation which is the final outcome 
of this work will, in principle, always yield some solution of the 
stability problem. The price that must be paid for assuring that 
the method always works is the restriction to a particular class 
of Liapunov functions 1 * 2 . These Liapunov functions are 
solutions of a partial differential equation which turns out to 
be the generalization of an analogous equation proposed by 
Zubov 3 . There exist obvious connections of this method with 
the problem of optimal control. 

The research was carried out while the author was visiting the 
Control and Information Systems Laboratory of the School of 
Electrical Engineering, Purdue University and was supported in 
part by the National Science Foundation Grant No. G-16460 
and in part by the Office of Naval Research Nonr-3693 (00). 


Nomenclature 

Standard vector notation is used with the following con¬ 
ventions. capital letters are matrices, small Latin letters are 
vectors, Greek letters and small Latin letters with subscripts 
are scalars. Exceptions: t, v 9 are scalars. 

The inner product of the two vectors x and y is denoted by (x, y). 

The transpose of a matrix A is denoted by A'. 

An equilibrium state that is stable, but not asymptotically 
stable, is called weakly stable. 

A scalar function 6 = 6 (x) is positive (negative) definite 
on the trajectories of a system in a region S czE n if 6 (x) ^ 0 in 
S (0(x) g 0) and 6 (x) =|= 0 on any non-singular solution of the 
system. By a Liapunov function is meant any scalar function 
which gives the answer to the stability properties of a solution 
of a system. 

Unless otherwise stated, it is assumed throughout that all 
the scalar functions used have continuous first partial derivatives. 


Liapunov’s Second Method 

Liapunov’s second method can be codified in a set of 
theorems 1 * 2 * 4 * which prove that if for a given system there 
exists a scalar function with certain properties, called a Liapunov 
function, then conclusions can be drawn about the stability 
properties of the solutions of the system. There also exist a set 
of inverse theorems 4 ’ 6 which guarantee the existence of such 
a function. 

Given the non-linear autonomous dynamic system 7 

*=f(x) f (0) = 0 (1) 

1 S a VeCtor " valued fun ction, the problem of the 
stability analysis of its equilibrium point is then formally 
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reduced to the search for a positive definite 8 scalar function 
yj = y) (x) and a scalar function v = v (x), v (0) — 0, such that 
the partial differential equation 

'l'(x)=- (grad V (x), f (x)) = £; —f t (x) (2) 

is satisfied. 

From the form of the scalar function v (x), conclusions may 
be drawn about the stability properties of the solution x = 0 of 
the system of eqn (1) and about the range of these properties 9 * 10 . 

The major problem is then to find a definite scalar function 
yj (x), such that the solution v = v (x) of eqn (2) satisfies the 
condition v (0) = 0. The inverse theorems 4 * 6 guarantee that 
such scalar functions ip (x) and v (x) exist. The stability problem 
is then reduced to the problem of finding necessary and sufficient 
conditions that a given scalar function ip (x) be positive definite. 
The existence and the possibility of using such conditions is 
however much in doubt. 

A different and more sensible approach is that of finding a 
sufficient condition which guarantees that a scalar function 
y) (x) is at least definite on the trajectories of system (1) and such 
that there always exists a function ip (x) satisfying this condition 
and a corresponding v = v (x), v (0) = 0 which satisfies eqn (2). 


Theorem 1 —If [i (x) is an integrating factor of the Pfaffian 
differential eqn (4) with solution v = v (x) and oc = oc (v) is 
an arbitrary scalar function, then pc (x) doc/dv is also an integrat¬ 
ing factor of eqn (4) with solution oc = oc (v (x)) = a* (x). 


It has been shown that the stability problem is reduced to the 
search for scalar functions v = v (x), v (0) = 0 and ip = ip (x) 
positive definite on the trajectories of system (1). Then the 
problem may be reduced to that of seeking a scalar function 
oc ( v ) such that 

^/i(x)m(x) = iKx) 

or 

da (: v ) \p_ (x) 

dv (x) / u (x) co (x) 

The functional eqn (11) can be readily solved if 


( 10 ) 

(ID 


fl(x)co(x) 


=J8 (»(*)) 


( 12 ) 


On the basis of these considerations the following procedure 
can be developed. 

Take a scalar function v 1 = v 1 (x), v x (0) = 0 and compute 
its total time derivative 


A Generalization of Zubov’s Equation 

In this section some remarks are made on the form of the 
scalar function ip (x), defined in eqn (2). These remarks will 
constitute the basis of the present method. 

The following question is posed. If an arbitrary scalar 
function u 1 = v x (x) is given and 


iAi. (x) = (grad v 1 (x), / (x)) 


(3) 


under what conditions on ipj (x) is it possible to compute from 
the scalar function v 1 = (x) a new scalar function = t? 2 W? 

such that ip* (x) - (grad v 2 (x),/(x)) has a certain required form? 
Consider the Pfaffian differential equation 

co (x) = (y (x), dx)=0 (4) 

Let ju (x) be an integrating factor of this equation, that is 

^ (x) y M = grad v (x) (5) 

The scalar function v = v (x) is then a particular integral of the 
Pfaffian differential eqn (4), that is, 

dv — (grad v (x), dx) (6) 

Consider now an arbitrary function oc = oc (v). From eqn (4) 
it follows that 

(7) 


k da, 


^ (x) (y ( x )s dx ) ^ 0 

By substituting eqn (5) into eqn (7) 


da 

dv 


(gradt?(x),dx) = 0 


which from eqn (6) is identically equal to 
da 


dv 


dv = da = 0 


These results can be summarized in the following. 


dv x 

dt 


= (grad 0c), / (x »=y ( x ), y (0) = 0 


(13) 


Next look for a scalar function y> (x ) which is at least definite 
on the trajectories of system (1) and a scalar function /? OO, 
(3 (s) ds < co such that 

' Hx) 




(14) 


■=j8(J*x) 


(15) 


y(x) 

Then the differential equation 

dot (Ui) _ »Kx) 

duj. 7 00 

can be integrated. Its solution <x 
such that 

a *=(grad a.* (x), f 00)=0 (O 

which is at least definite on the trajectories of the system (1), 
and because of the assumptions made on oc (v^) and v t (0 solve 
the stability problem. 

Substituting eqn (14) into eqn (13) gives 

(grad Uj (x), / (x)) = ^ ^ 


■ oc(Vj) = oc* (x) will be 
(16) 


(17) 


( 8 ) 


(9) 


rauiqvA/,./ Wffjipj) 

which is a generalization of Zubov’s eqn 8 : 

(grad v, f (x)) =<j)(x)(l+v) (18) 

The stability theorem deduced from eqn (18) may be stated as: 

Theorem 2—The stability problem of the solution x = 0 of 
the system eqn (1) is reduced to finding scalar functions (x), 
y, (x), p ( Vl ) such that v, (0) = 0, (?) ds < oo and y (x) is 

definite on the trajectories of the system of eqn (1). 

Integrating eqn (15) 

aOT) = J%(s) ds ( l9 ) 

from which is deduced 
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Theorem 3 —The solution x = 0 of the system of eqn(l) is 
asymptotically stable in a closed, bounded region S: x* (x) g d 9 
S > 0, if there exist scalar functions v 1 (x), ip (x), ft (y x ) satisfying 
the following conditions: 

(0 (0) = 0. 

0‘0 V (x) negative definite on the trajectories of the system 
eqn (1). 

0*0 p* ft (s)ds < co. 

0*0 x* (x) = /ji p (s) ds > 0 in 5, x # 0, x* (0) = 0, 
and such that eqn (18) is satisfied. 

Corollary I—The solution x = 0 of the system of eqn (1) 
is asymptotically stable in the large if all the conditions of 
Theorem 2 are satisfied and 

lim a*(x) = lim /?(s)ds=oo 

||a:|| -+ co J 0 


v 2 (0) — 0. In particular if the system of eqn (1) is asymptotically 
stable v 2 (x) is definite and 6 (v 2 ) may be chosen so that 6 (v 9 )= Iv 
Re{l} < 0. 2 ’ 

Proof —This theorem may be proved directly following a 
method used by Vrkoch 12 , or by using the inverse theorem 4 * 6 . 
A proof is sketched using this latter way for the case of asymp¬ 
totic stability. 

Given any Liapunov function v = v (x), v (0) = 0, v (x) # 0 
for x 0 , v (0) = 0, this function represents a hypersurface 
in E n+1 with a strong minimum for x = 0. 

Take any section of this hypersurface with the hyperplane 
v = const ^ 0. This section is a closed bounded hypersurface. 
Represented in parametric form 

= (22) 

Since v (x) > 0 for x # 0 it is possible to construct a unique 
integral surface S = v* (x) of the equation 


Corollary 2— If all the conditions of Theorem 3 are satisfied 
with the sign of ip (x) changed, then the solution x = 0 of the 
system of eqn (1) is completely unstable 11 . 

Remark 1 —It is always possible to give sufficient conditions 
for complete instability, from any theorem on asymptotic 
stability. In the following sections are presented numerous 
theorems on asymptotical stability, and it is always implied, 
even if not explicitly stated, that a similar theorem for complete 
instability holds. 

Since, given any scalar function v 2 = v 2 (x), it is always 
possible to find a functional Q = Q (v 2 (x)), Q (0) = 0 such 
that the scalar function Q = D* (x) = Q (v 2 (x)) is semi-definite, 
the following simplified procedure for constructing Liapunov’s 
functions can be developed. 

First, seek a scalar function v 2 = v 2 (x), v (0) = 0, such that 
dv 2 

77 =(grad v 2 (x), f (x)) = Q (v 2 ) (20) 

where 6 = 6 (v 2 ), is a bounded scalar function. Eqn (20) is a 
special case of eqn (17). 

Then it is always possible to integrate the equation 


d «2 0 2 ) 
dv 2 



and its solution a 2 — oc 2 (v 2 ) = « 2 * (x) will be such that 


( 21 ) 


=(grad a* (x), f (x)) = £2* (x) 


where Q* (x) is semidefinite. If no degeneracy occurs and if 



then the scalar function * 2 = « 2 * (*) is a Liapunov function 
of the system of eqn (1). Theorems 1 and 2 and the corollaries 
apply with minor changes to this case. 

The following theorem answers the question of the existence 
of solutions of eqn (20). 


Theorem 4 —There always exists a scalar function 
that eqn (20) has a solution which satisfies the 


6 (v 2 ) such 
condition 


v* = (grad V* (x), f (x)) =-v* (x) (23) 

going through the hypersurface (eqn (22)). 

Consider the characteristic system of the partial differential 

' ,n(23) d„* iXt A 

"*• m- it ‘- l .” (24) 

Since the system of eqn (1) is by assumption asymptotically 
stable, the solutions x^ = x* (/), v* ~ v* (t) of the characteristic 
system tends to zero as /-> co. Hence v* ( x )| Ar . 0 = 0. 

Remark 2 —An existence theorem analogous to Theorem 4 
may be proved also for the partial differential eqn (17). It is 
seen that the necessary and sufficient conditions that the scalar 
function ip (x) must satisfy for eqn (17) to have a solution 
v i ~ v i (x) such that v x (0) = 0 are not identical to the con¬ 
ditions on 6 (x) in Zubov’s eqn (18). In this latter case 
Io0 (x ( r )) dr < co, a relation which in this case is only sufficient. 

It is worthwhile to emphasize the major differences between 
eqns (17) and (20). As previously pointed out the ^-functions 
^ = x* (x), obtained from eqn (18) are Liapunov functions for 
the system of eqn (1). The only possible degenerate case, in 
which x is semi-definite, will never arise. 

In fact if x = x* (x) is continuous and semi-definite then it 
has a strong minimum on the manifold M on which x* (x) = 0. 
Hence grad x* (x) = 0 on M and M is an integral manifold 
of system (l) 3 ’ 9 . This contradicts the hypothesis that ip (x) is 
definite on the trajectories of system (1). This reasoning does not 
appiy to eqn (20) where Q* (x) is only semi-definite. Then 
x* (x) = (x) = 0 on some integral manifold N. All the 

information obtained in this case concerns the stability of the 
manifold N and not of the equilibrium point. In some cases 
then the procedure must be repeated in order to find a new 
a i = V (x) satisfying an equation of the type of eqn (17) which 
may be semi-definite as long as there does not exist a point 
x = x, # 0 on which x* (x e ) = x x * (x e ) = 0. In this case one 
not only obtains information about the stability properties of 
the equilibrium point, but one is also able to find some integral 
of the system. The same situation may arise from eqn (17) if 
W (x) is a semi-definite function. 

Eqn (20) is very important in itself since its solutions are the 
so-called isochrones of system (1). The knowledge of the iso¬ 
chrones gives important information about the qualitative 
behaviour of the solutions of the system of eqn (1). 
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Remark 3 —For some systems it may happen that the scalar 
function f (x) in eqn (17) or Q* (x) in eqn (20) is identically 
zero. The scalar function a = a* (x) is then a first integral of 
the system. 


A Useful Change of Variables 

The method for constructing Liapunov functions developed 
in the previous section contrasts strongly with the methods in 
use up to now. The essence of this method is the introduction 
of the functions ft (^) and 6 (l\ 2 ) respectively in eqns (17) and (20). 
The major step is now to find a scalar function v = v (x), 
v (0) = 0, such that dv/dt has the form 0 2 (v) or yj (x) (v) 9 but 
is otherwise completely arbitrary. 

Regarding the problem of constructing a Liapunov function 
by solving a partial differential equation it is seen that the 
original linear partial differential eqn (2) (with unknown right- 
hand side) has become a quasi-linear partial differential eqn (17) 
or (20) whose right-hand side has a certain well-defined form. 

In this paragraph is performed a particular transformation 
of variables x -> z. One of the components of the next state 
vector z is the scalar function v. 

The stability problem will then be reduced to a search for 
a scalar function £ = £ (z), satisfying a certain non-linear 
partial differential equation, whose right-hand side is any 
definite function which depends on only one particular com¬ 
ponent of the vector z. 

Consider the scalar equation 

w = v(x) u(0) = 0 


which is equivalent to the equation 

Xi = £i(Xl> x i- 1> x i+1’ 

Since this operation is performed only once in the whole 
procedure the notations can be simplified. Introduce the 
n vector z defined as follows 

z k -x k fc# i Z; = w (26) 


Eqn (32) may be written 

Xi = {;(Z) C 27 ) 


so that v (x 1 , 


and finally 


Xi— 1, Si (^)j Xf-\--y , . . < 

■j x ri) W 

= 0 from which 

dv> dv ___Q 

0X; dX:dX: 

(i#i) 

(28) 


dv_ 0^ 
dx- t 0w 


0o, iKi 

0X; / 0W 


00 _ 0 /9Ci 

OXj azjl 0w 


0 ^ j) 


(29) 


The transformation of coordinates 
x-*-z,&(z) 

can now be performed on eqns (17) and (20). 


(30) 


These equations take respectively the form 




and 


_ 

(i#j) 

(31). 


(i^j) 

(32) 


1 02 j /1*1={l 

Eqn (32) is of particular interest and can be written as 



0*1) 


(33) 


In this latter equation the usefulness of the present method is 
much emphasized. 

It can be seen that the only requirement is that the right-hand 
side of eqn (33) depends only on w 2 . If the function fi (w 2 ), 
defined by eqn (21) satisfies the condition J v Q 2 fi 2 (s) ds < co,the 
problem of the stability of the solution x = 0 of the system 
under investigation is solved. If one cannot find a scalar function 
a (w 2 ) such that a (w 2 )/0 (w 2 ) is bounded, then it is still possible 
to study the stability of some first integral of the system, going 
through the origin. 

In other words, whatever the function 6 (vv 2 ) is it will always 
be possible to have some answer about the stability properties 
of the system. 

Unfortunately the solution of the partial differential equation 
is not a simple matter and it is possible to integrate it explicitly 
only if it is possible to integrate its characteristic system. A more 
reasonable approach is to choose a suitable form for the unknown 
scalar function Si — Si (x), (0) = 0 having a certain number 

of known coefficients, then compute the unknown coefficients 
in such a way that the right-hand side of eqn (33) is a function 
which depends only on w 2 . 

The possibility of doing this depends of course on the right 
choice of the form f*. Although numerous examples have been 
solved, no general information about suitable forms for ti is 
available. 


Examples 

In the case of non-linear systems, if one is not able to inte¬ 
grate the characteristic system of eqns (31) or (32), then an 
alternative procedure is to look for a suitable form of the 
unknowns v = v (x) or Si (z, w) which allows a separation 
of variables. 

Consider for example the system 


for which 
and 


x — y 

y — — ay — ax 3 — 
v x = ax+y 
v x = -x 2 v l 


(34) 

(35) 

(36) 


It is seen that the solution y = - ax is asymptotically stable. 
By integrating it one obtains x — x Q exp (— at). 

It is concluded that the equilibrium point x = y = 0 1S 
asymptotically stable for a > 0 and unstable for a < 0. In this 
particular case the stability problem is solved directly from 
eqns (35) and (36). This is not always the case. 
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Consider for example 


which may also be written in the usual form 


x=y 

y = ax + ax 2 y-y 3 -y ( a > 0) 
Assume v = ax 2 - y 2 then v = 2 y 2 (1 - v) 
from which no deduction can be drawn. Choose 


then 


\l/(x)=2y 2 (l-v) 2 


'v 

J 0 


a O) = /?(s)ds = 


(1 — s) ds = 



(37) 


v=2x 2 — y 4 

and it may be checked that 


v=6(w) = —2v 


It is concluded that the solution 2 x 2 = / is globally 
asymptotically stable. 

By direct analysis of this particular solution one can show 
that the solution x = y = 0 of the system eqn (38) is not 
globally asymptotically stable. 


CC* (x) = (ax 2 -y 2 )-~ (ax 2 -y 2 ) 2 
a* (x) = 2y 2 (t — v) 2 

from which it is concluded that x — y = 0 is unstable. 

The next example illustrates the advantage of eqn (33) with 
respect to the other formulations. Consider the system 


y=x-L y 

for which eqn (33) takes the form 


Assume 


(39) 

£=«!>+/oo]* 


(40) 

for which eqn (39) becomes 



2 

~y 3 C ™ + f(y)¥-2w-2f(y) 




Conclusion 

The general problem of stability analysis of the equilibrium 
point of a control system, represented in the form of eqn (1) 
has been solved. 

This general problem has been reduced to the integration 
of partial differential eqns (20), (31), (17) or (33). 

The close-form integration of eqns (20) or (31) presents, in 
the case of non-linear systems, the same difficulties as the 
equations which arise from optimal control problems; hence 
in most of the cases numerical methods must be applied. On the 
other hand, for expressions of the form of eqns (17) and (33) 
various examples with closed-form solutions have been found. 
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dy 


from which may be set 


0(w)= -2 w 

(41) 

- 2 /w 4 ,|z.„ 

(42) 

Jy-.S£-o 

a 0y 

(43) 


Integrating eqn (42) gives 


f(y)=±y 4 

and from eqn (43) f(y) = / and a 2 
results into eqn (41) gives 


(44) 

= }. Substituting these 


{=A(»+/)* 


(45) 
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NEW METHODS FOR CONSTRUCTING LIAPUNOV FUNCTIONS FOR TIME-INVARIANT CONTROL SYSTEMS 


DISCUSSION 


M. Mansour, Institut fur Automatik und Industrielle Elektronik ETH, 

Altbachstrasse 531, Dietlikon ZH, Switzerland 

(1) For the stability study of the system 

x=f(x) (1) 

/(0) = 0, the main problem is to find ip (x) & V (x) such that ip (x) is 
definite &V(x) with V(0) = 0 satisfying the basic partial differential 
equation ~ y 

^(x)=VF-/(x)=X 5 -/iW i = l-n (2) 

Eqn (17) in the paper can be obtained directly from the above equation 
by assuming r v 

V= J J5(s)ds 

which can be always satisfied by V, v. Eqn (20) can be obtained 
directly from (17) by assuming ip (x) = function of v = 0(v) 

(2) By using eqn (17) the problem of finding a Liapunov function 
for the study of the equilibrium of the origin x — 0 is not solved as 
the difficulty of finding two functions V (x) and ip (x) satisfying the 
basic partial differential equation is not transferred to finding three 
functions v (x), ip (x) and p ( v ) which I think is still difficult. 

(3) As to the example given by the author, the stability or instability 

of the origin may be detected easily by considering the first approxima¬ 
tion. As the right-hand side of the system equations in the three 
examples is composed only of terms of the first and third degree it is 
possible to obtain a Liapunov function by solving eqn (2) assuming 
ip (x) - ip 2 (x) + (x) and V(x) = V 2 (x) + F 4 (x) where ip 2 , V 2 

contain only terms of the second degree and %, V i contain only terms 
of the fourth degree. 

(4) Using the Zubov method one can, under certain conditions, 
find the region of asymptotic stability using his construction procedure 
either analytically or with the aid of a computer. I do not think that 
it is possible, using the equations derived by the author, to obtain 
such a region. However, this method may give information about 
the stability of some particular solution of the system equations. 

G. P. Szego, in reply 

I am very grateful to Mr. Mansour for his interest in my work. The 
disagreement between us is mostly due to misinterpretation and, 
maybe, incomplete statements of the purpose of this paper. I shall 
clarify this purpose and present a new theorem that was only men¬ 
tioned in the paper, and which I hope will emphasize the most impor¬ 
tant points of my contribution. 

The main purpose and the main result of my work is an extension 
of the class of stability functions beyond the realm of the classical 
theory. This new class of functions will obviously apply to the investiga¬ 
tion of the stability properties of invariant sets of (1) which are larger 
than equilibrium points. These results will be helpful in the case in 
which an extremely fine problem of synthesis of sub-optimal control 
systems exists. 

I present the main theorem as follows 1 . 


Theorem 

Consider the dynamical system (1). Let 

(i) v (x) be a scalar function s C l in the whole space E n , 

(ii) yj ( v ) be a scalar function e C° in the whole space E n , 

(iii) M be the manifold on which v (x) = 0. 

Assume that 

( iv) ip ( v(x )) s 0 in all points of M, ip (v(x)) 4= 0 for x $ M, 


( v ) The equation 

< grad v (x)J(x) > = - \jj (a) (1) 

is satisfied in the whole space E n . 

(w) v(x)il/(v(x))> 0 (2) 

in the whole space E n . 

(vii) The trivial solution v = 0 of the equation 

v=-ij/(v) (3) 

is globally asymptotically stable. 

(viii) a (p (x, Mj) <\v(x)\<b(p (x, Mj) (4) 


where q (x, M) is the Euclidean distance of a point x from the set M, 
the scalar functions a (r) and b (r) are positive definite and a (r) is such 
that lim a(r) = oo. Then M is globally asymptotically stable. 

Weaker versions of this problem may be proved for the case of 
stability and quasi-asymptotic stability; in particular, conditions (3) 
and (4) will be relaxed. Condition (4), which imposes the complete 
identity of the v norm with the Euclidean norm is required for the 
case that M is non-compact which is indeed the case of interest. 
However, when additional informations of the flow may be obtained 
this condition can be relaxed from either side (as in the examples of 
the paper). 

I hope that this theorem will clarify and strengthen the results 
of the paper. 

As to Mr. Mansour’s second and third remarks, it must be clear 
by now that they are obviously formally correct for the case of stability 
of equilibrium points, but they do not apply to the general problem 
studied in the paper. 

I do not agree with the fourth remark since I have shown that 
Zubov’s equation is a particular case of the equations derived in the 
paper. 

The first remark is formally correct. I do, however, believe that 
the derivation of eqns (12) and (20) as done in the paper is worth the 
trouble because it shows the reason for the assumptions of theorems 
(2) and (3). 
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tial equation for the stability analysis of time invariant control 
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A Method of Investigating Stability 

H. H. ROSENBROCK 


Summary 

A method is given for considering the stability of the trivial solution of 
x — A (t, x) x. 

It is shown that if x is directed towards the interior of a certain 
region H (bounded by hyperplanes) at each of its vertices, whenever A 
is within some closed region G (of the space E 2 of the elements of A) 
then, subject to A (r, x) E G for all t > t 0 and all x, the solution 
x = 0 is generally asymptotically and uniformly stable . Some applica¬ 
tions are considered. 


Sommaire 

On donne une methode d’etude de la stability de la solution du type 
d’equation: 

x = A(t,x)x . 

On montre que si x est dirige vers l’interieur d’une certaine region 
H (bornee par des hyperplans) a chacun de ses sommets, chaque fois 
que A est dans une region fermee G (de 1’espace E 2 des elements de 
A), a condition que A ( t , x) 6 G pour tout t > t 0 et tout x, la solu¬ 
tion * = 0 est generalement asymptotiquement et uniformement 
stable. Quelques applications vont 6tre examinees. 

Zusammenfassung 

Der Aufsatz behandelt ein Verfahren zur Untersuchung der Stabilitat 
der trivialen Losung von 

x = A(t, x) x. 

Es wird gezeigt: wenn a: in einem bestimmten (durch Hyperebenen 
begrenzten) Bereich H an jeder seiner Ecken nach innen gerichtet ist 
und wenn A sich innerhalb eines abgeschlossenen Bereiches G (des 
Raumes E 2 der Elemente von A) befindet, dann ist unter der Be- 
dingung A ( t , x) EG fur alle t > / 0 und alle x die Losung x — 0 all- 
gemein asymptotisch und gleichmaBig stabiL Einige Anwendungen 
werden besprochen. 


Introduction 


In an earlier paper by the author 1 some results were given 
concerning the stability of a second-order differential equation 
The method of proof used there can readily be extended in the 
following way. 

The following equation is given: 


x=A(t, x)x (i) 

where x is an w-vectorand A satisfies conditions ensuring that a 
umque solution passes at time / 0 through any jc 0 in some region R 
of the space (x). Let G be a closed bounded region in the space 
01 e l® m ents of A. Let H be a closed region in R, such 
that every point of its boundary lies in at least one of a given 
set of hyperplanes and such that every ray from the origin 
intersects the boundary of H once and only once. (The hyper- 
planes will be n + 1 or more in number.) Let the points x = h 
e the vertices of H, and at each such point let x be non-zero and 
directed out of the region exterior to H and into the interior 


of H for all A e G. Then every solution of eqn (1), starting at 
time t 0 from some x 0 6 H , is uniformly, asymptotically stable 
subject to the condition that A (/, x) e G for all t > t 0 and all 
xeR. If R is the whole space (x), the uniform asymptotic 
stability holds in the large. 

To prove this, let n be the unit inward normal on one face 
of the region H. Then n'x is positive everywhere on this face 
when A e G. For suppose to the contrary that at some point x 
in the face n'x < 0 for some A 0 e G. The vector x can be 
written 

**2>A (2) 

j 

where the sum is taken over all the vertices lying in the face 
considered, every ocj > 0, and some « 3 - >0. 

Z^n'A o hj<0 ( 3 ) 

j 

contradicting the assumption that every n'x is positive at the 
points hj for all Ae G. 

Let r = 1 be some linear dimension of H, and consider the 
set of similar regions H(r), r < 1. At corresponding points on 
the boundary of each such region x/r and n have the same value 
for given A 0 e G, and n'x/r is positive. For given A 0 and r, and 
considering all points lying in the boundary of H (r), there is 
a least value e(A 0 ) > 0 of n'x/r, which is achieved at one of the 
vertices h, (r). The continuous function e (A a ) is positive for all 
A 0 6 G, and hence has a least value e > 0 at some point in G. 
Thus at every point on the boundary of H (r), n’x > sr > 0 
for all A e G. 

Now let V (x) be a function defined so that V(x) —r 
when x lies in the boundary of H (r). Evidently K(x) > 0 
when x =h 0, and V(0) = 0. The function V(x) has a one-sided 
derivative in every direction at the boundary of H[V{x)], and 
this derivative is negative along any direction entering H. 
Hence Khas a right-hand derivative V (possibly discontinuous) 
along any solution of eqn (1) and the condition n'x > er > 0 
implies V < - rjr<0. Then since r = V( x ) we have, for all 
e ’ f —. — V ^ ft follows that V is a Liapunov function for 
eqn (1) subject to A e G, and all solutions starting in H(l) tend 
asymptotically and uniformly to x = 0 as t -> oo. The region 
H a) is then to be chosen as large as possible while remaining 

Examples 

Let the vertices of H (1) be at unit distance from the origin 
along each (positive and negative) coordinate axis. When x, = 1 
and x,- = 0, i + j, 

i=l,2, ...,n 

&ij\ < —£j<0 ( 4 ) 


a u + 'L I 

i 

i*j 
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it is easy to see that x is directed towards the interior of H. If (4) 
holds for all j, and if 

\ajj\<P, j = (5) 

then A is restricted to a certain closed bounded region, G, in the 
space E n i. For all A e G the conditions of the previous section 
are fulfilled. Hence if conditions (4) and (5) are satisfied for all 
t > t 0 and all x, the trivial solution x = 0 is asymptotically and 
uniformly stable in the large. A Liapunov function for the 
system is F=£|x ; | (6) 

i 

This result may be compared with similar results obtained 
elsewhere 2 * 3 by a different method. The earlier results were 
complicated by the need to allow some of the e j in (4) to be zero. 
When this happens the present method allows only stability, 
and not asymptotic stability, to be asserted. The equations of 
many physical systems involving the diffusion of heat, matter, 
electricity, etc., obey the relaxed form of condition (4) used in 
previous papers by the author 2 * 3 . 

As a second example, let the vertices of H{ 1) be at the 

2 n points x 1 =±l,x 2 =±l,...,x„=±l (7) 

so that at a vertex 


Now take the vertices of H (1) at the 4 n points 

( + a, +1, 0, 0,...,0) 

( + oc, 0, ±1, 0, ...,0) 

( + oc, 0, 0, ±1, ...,0) (14) 


(+a, 0, 0, 0,...,+1) 

At the In vertices where £ = oc > 0, 

Z=-roc±gi, i = l, 2,ft 

which will be negative if 

y —i = l,2,...,ft 

r>Pi>0 

Pioc^y 


(15) 


(16) 


At the same points 

yi=hiOC-AiSgnyi (17) 

and (i,yY will be directed towards the interior of H at each of 
these points if in addition to (16) 



Xi = Y a u x i = s § n x i+ I a ‘j sgn x > 

j J* i 

(8) 

If 

a lt + Xi«ul^- £ i <0 > i = l,2,...,n 

j 

(9) 


the velocity x will be directed towards the interior of H at every 
vertex. Then, if the a iS (t, x) satisfy (9) and (5) for all t > t 0 and 
all x, the trivial solution x = 0 is asymptotically and uniformly 
stable in the large. A Liapunov function is 

F=max|x ; | (10) 

i 

For the third example consider a plant which can be described 
by the linear equation with constant coefficients 

x = Bx (U) 


For simplicity suppose that every root A of | Bu +-Aw | = 0 is 
real, simple and positive. This is a common situation in some 
chemical engineering applications 4 . Let the plant be controlled 
in the manner suggested by the equation 


( 





( 12 ) 


where r 0 , g 0 and h 0 are functions of t,%, x. Make a transforma¬ 
tion of variables from £, x to £,y in such a way that B is brought 
to diagonal form and 



Here the r, g t and are functions of t, C, y. 


i 1,2, ...,n 

i 


(18) 


The rema inin g In vertices give no further condition. Now 
choose « = yip!, which satisfies the last of inequalities (16). 
The conditions 


I £il + ^1 ~ 7 j i — 1,2,..., ft j 
£|hj|+£ 2 <>7; e 2 >0 

i 

0 <p t <r<p 2 I 


0<p 1 <2 i <p 2 j 1 = 1,2,...^ 

yr\<liiPx 


(19) 


restrict the matrix in eqn (13) to a closed region G in E( n + i ) 2 . 
If they are satisfied for all t > t 09 £,y, the solution £ = y == 0 
is asymptotically and uniformly stable in the large. 

The system corresponding to eqn (12) is shown in Figure 1. 
It is a generalization of one considered by Lurie 5 and by Letov 6 , 
their corresponding system being obtained by putting 


go (t, 6, X) x - r 0 ( t , l, x) { =/ (e'x - k& 
h 0 (t,£,x) = d 


( 20 ) 

( 21 ) 


for some constant k, c, d. The important condition among 
inequalities (19) is the last, which shows that the system is 
always stable provided that r is sufficiently large for all t,% and x. 
This agrees with the results of Lurie and Letov, but (contrary 
to a statement of LaSalle and Lefschetz 7 ) larger values of r 
imply looser control. 

Conditions (19) are in general more restrictive than those 
obtained by the methods of Lurie or Letov (where those apply). 
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For example, because H was chosen with a particular symmetry, 
conditions (19) make no distinction between positive and nega¬ 
tive values of g t or h { . Systems stable according to (19) will 
therefore remain stable when the sign of the feedback around 
the plant is reversed. 



Figure 1 


are two known conditions of this type. Inequalities (19) represent 
a new result of the same kind which may be written, with 
reference to eqn (13), 


(max |gi|)£ \hi\ 


r >—-—:—p >0 

mm A { 


The theorem which has been proved may therefore be regarded, 
from one point of view, as a device for generating such con¬ 
ditions. 

In many practical situations the equations of a system can 
be replaced in any one working condition by a linear auto¬ 
nomous approximation, but the appropriate linearization 
changes as the working conditions change. If the system is to 
be stable in each fixed working condition the A must satisfy the 
criterion Re A > s > 0. It would be desirable to have additional 
criteria for the A which ensured that the system was stable when 
the working conditions were changing. Such results can be 
obtained by the present method 1 , but only when A has a special 
form. Since A has « 2 elements, conditions on the A cannot 
generally restrict A to a closed region G: for this to be possible^ 
must have no more than n independent variable elements. 


A more judicious choice of H would no doubt give better 
results at the cost of some increased complication. Such improved 
results would still be expected to give more stringent conditions 
than are obtained by Lurie and Letov from an initially more 
restricted class of systems. 

Comments 

Stability criteria of the type considered restrict the elements 
of A by the condition AeG, but leave the behaviour of the 
elements otherwise unrestricted. It is then possible for A to 
assume a constant value A 0 e G. The necessary and sufficient 
condition for stability in this case is that every Re A (A 0 ) > e > 0, 
where A is defined by 

\A o u + Au\=0 (22) 

Thus it follows that the condition AeG must ensure that every 
Re A (A) > e> 0. Inequality (4) (for all j) and inequality (9) 
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DISCUSSION 


M. L. Deutsch, Research Department , Socony Mobil Oil Company , 

Inc., Paulsboro, New Jersey , U.S.A. 

For this discussion, regions such as Rosenbrock’s H are called 
‘starlike’. It appears that the crux of the matter is convexity rather 
than starlike character. For if we properly connect the vertices of H by 
hyperplanes, we produce a convex polyhedron, P, such that x eH = 
xEP, and to which the results of the paper apply. The application 
is then in some sense even more general, because the requirement of 
an inward-pointing X at each of the vertices of P is less restrictive 
than the requirement of such a set of x’s at the vertices of H 
{Figure A.). 

At V, for example, if x lies in H it certainly lies in P, but not con¬ 
versely, The region P thus defines a good Liapunov function. 

Further, Rosenbrock’s conditions do not guarantee stability in 
all starlike regions, H. Consider Figure B. 


For this starlike region, H lt the condition that X points inward 
at Vj and V 2 does not guarantee that x points inward along the line 
v i V 2 . But the line V x V 3 converts the starlike region, H x , into a 
convex polyhedron, to which Rosenbrock’s conditions and results 
apply. 


V 



Figure A Figure B 
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H. H. Rosenbrock, in reply 

I am indebted to Dr. Deutsch for pointing out an error in the paper, 
which has since been corrected. The important condition in the proof 
is that n'x should be positive on every face at each vertex of H. This 
condition was equated in the paper with the condition that n should 
be directed towards the interior of H at each corner. At a re-entrant 
corner this is not sufficient: the velocity must be directed out of the 
region exterior to H. 

With the amendment the result in the paper is correct for star¬ 
shaped regions. This will give no further information about stability 
than the corresponding convex regions, and the latter will be simpler 
to handle and therefore generally preferable. There may, however, 
sometimes be an advantage in using a star-shaped region, as this will 
give more information about the trajectories than can be obtained 
from the convex region. 


P. C. Parks, Department of Aeronautics , The University , Southampton , 
England 

There is a connection between the author’s treatment of the system 
X = A x with his Liapunov function and the so-called Gershgorin 
theorem about the location of the latent roots of A. This theorem 
states that the latent roots are situated within the union of circles with 
centres an and radii 

I \ a u\ i = U2...n 

J= i 

(or alternatively, of radii 

Z K-;l) • 

j =1 

The author’s criterion, obtained by Liapunov’s second method, 
specifies that one of these two unions of circles lies within the left-hand 
half of the complex plane. 


H. H. Rosenbrock, in reply 

The theorem quoted by Mr. Parks does indeed give the ‘known 
conditions’ mentioned after eqn (22) of the paper. 


M. Mansour, Institut fur Automatik und Industrielle Elektronik, 
Altbachstrasse 531, Dietlikon ZH, Switzerland 

My first remark deals with the statement in the second paragraph 
‘every ray from the origin intersects the boundary of H once and only 
once’. My idea is that it is not sufficient, and in order to prove the 
theorem this may be replaced by the statement ‘any ray must not 
intersect the boundary of H more than twice’ to avoid the possibility 
of n'x to be negative at some points on the hyperplane. This pos¬ 
sibility was also pointed out by Dr. Deutsch in his discussion. 

My second remark is that the hypersphere 

X 1 +X 2 + ...X 2 =r2 

can be considered as a limiting case of some closed region bounded 
by hyperplanes. The sufficient conditions for asymptotic stability of 
every solution of X — A ( t , x ) x starting at time t 0 from some x 0 Eff(r) 
is that at each point of the boundary 

x\-\~x^ + ...x^ = r , 

X is not zero and directed into the interior of H (r) for all A G G. For 

a hypersphere this condition is equivalent to x'x negative. 

i.e. 

X 1 X 1 +X 2 X 2 + ■■■X n x„<0 


Let A be the matrix 


ni a i2 ■ 


l ln 


K &nl ^n2 

where an is a function of t and a i,j — 1 ... n 


x 1 x = x 1 ‘ 


a n 

^12 

2 

a ln + a ni 


a l n JrCl n'l 


2 

a 22 


<x=- 


- x l Cx 


This is a quadratic form whose matrix is C. For x 1 x 1 to be negative 
definite the matrix C must satisfy the Sylvester inequalities 


— a xl >0 


~ a n 

a i2 J r a 2i 

2 

a 13 + ^31 


-^11 
#12 ^21 


a 12 + ^21 


^12 + ^21 


— a 


22 


2 

“ a 22 

a 13 + a 31 

2 

^23 + a 32 


>0 


<^ 23+^32 ) 

2 ) 


— a 


33 


>0... etc. 


These are sufficient conditions for the stability of the considered 
system. The above is equivalent to taking a Liapunov function 

V = x 1 E x where E is the unit matrix 

V — — 2 x 1 C x where — 2 C — A -|- A 1 
Applying this to the third example in the paper gives 

This condition seems to be more flexible and gives better results, 
especially when gi + h% cancel to some extent. However, this is not 
always the case. 


H. H. Rosenbrock, in reply 

The condition suggested by Dr. Mansour is that any straight line 
not in the boundary of H should, at most, intersect it twice. This is 
equivalent to convexity. It would also be necessary to ensure that H 
enclosed the origin. With this change in the conditions of H the 
original wording of the paper, mentioned in the reply to Dr. Deutsch, 
would be correct. 

The relative efficiency of different types of Liapunov functions 
depends on the problem considered. The result (19) of the paper, as 
stated therein, is not a good one. It could be improved by a better 
choice of H, but might still be worse than the result obtained by 
Dr. Mansour, using Krasovskii’s method. The result in (4), on the 
other hand, is better for many physical systems than results obtained 
by Krasovskii’s method. 
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GENERAL DISCUSSION OF STABILITY PROBLEMS 


J. P. LaSalle 

We all agree that the only method available for studying the stability 
of non-linear systems, which effectively takes into account the non- 
linearities of the system, is Liapunov’s second method. We have, 
however, a great deal of theory, and many special problems have 
been solved. I think that the most important problem for today is to 
find a means of making use of the large-scale computers which are 
available for applying Liapunov’s second method to control problems 
in a more effective way. 

I personally do not see any sign that we are close to a solution of 
this problem. 


G. P. Szego 

I certainly agree with Dr. LaSalle that the solution of the stability 
problem of control systems lies in the use of digital computers for 
constructing Liapunov functions which describe the behaviour of the 
system in the whole space. 

I would like, first of all, to justify the use of a computer for this 
particular task. It has been said that with a computer one can, after 
all, integrate the equations. Unfortunately one would have to integrate 
the differential equation for all initial conditions. On the other hand, 
for a Liapunov function which defines the behaviour of the system 
in the whole space the computation will be performed once and only 
once. 

There are mainly two different ways of approaching this problem. 
One is to solve a partial differential equation of the Zubov type or 
one of the partial differential equations I have presented in this section 
which will cover a wider range of problems. Of course, in this approach 


one has to make the assumption that there exists a polynomial 
Liapunov function which satisfies the partial differential equation. 

Assume now that we are able to do this, or better, that the com¬ 
puter is able to do this: the stability problem is not yet solved. The 
computer will give a polynomial form in n variables of order m. The 
problem is now reduced to the analysis of this form in order to decide 
what are the stability properties of the system. This task is not a 
trivial one. 

Another way of attacking this problem is to follow the method 
presented by Dr. Nesbit in a previous session. This method, however, 
also presents some unanswered questions in the case in which 
the non-linearities depend upon many components of the state vector. 
Both approaches seem to be feasible; the choice of one or the other 
strictly depends upon the particular problem in question. If one has 
to investigate a first system, or a well-defined equation, one has to use 
the first approach. If one is interested in the relative stability properties 
of one system with respect to the neighbouring ones, very likely the 
second approach will be more convenient. 

H. H. Rosenbrock 

I agree with Dr. LaSalle that it would be most attractive to use a 
digital computer to generate Liapunov functions. The method which 
I thought of trying was to specify the corners of H and allow the 
computer to examine each face at each corner. It would have to do 
this for all matrices in G, but this would be feasible if G were, say, 
a line. The computer would probably find that the velocity was 
directed out of H at one or more corners. Then it would be necessary 
to have some rules by which the computer could move the corners in 
order to improve the position. One might also perhaps allow the 
computer to put in fresh corners. 
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SYSTEM DYNAMICS AND OTHER PROBLEMS 


Process Dynamics and its Applications to Industrial Process 

Design and Process Control 

A Survey by T. J. WILLIAMS 


Introduction 

In introducing the subject of process dynamics, one should 
perhaps first define the area to be covered. While process 
dynamics has been used by others to cover a wider area, I would 
like to confine my remarks to topics related to the so-called 
process industries. That is, such industries as chemicals, petro¬ 
leum, glass, cement, metals, food, and related industries where 
chemical and mechanical changes are brought about in a 
continuous or semicontinuous stream of raw materials in order 
to produce a product or products whose bulk chemical or 
physical properties are the important source of their economic 
value. These remarks will also apply generally to much of the 
energy conversion industries who transform chemical or poten¬ 
tial energy into the more generally usable electrical and thermal 
forms. These remarks will not apply generally to those manu¬ 
facturing industries which produce a class of discrete objects as 
products, the individual properties of which objects are the 
source of their econo.mic value. 

The study of process dynamics can serve as a source of 
Intellectual satisfaction for the new knowledge obtained. It can 
also provide the possible remedy of an existing plant control 
problem. However, the major portion of all of our studies of 
process dynamics, or of the time dependent responses of our 
processes, should have as its ultimate purpose that of aiding 
the future design of new and/or improved control systems for 
our processes or of aiding the future design or operation of these 
processes themselves. 

The basic need in plant engineering is for sufficient knowl¬ 
edge of industrial processing plants, their manufacturing tools 
and machinery, and their related distribution and economic 
systems, so that one can theoretically characterize and compute 
a satisfactory representation of their dynamic behaviour in the 
face of any postulated external or internal influence. In addition, 
one must be able to do this in the design stage of a new 
plant or while a new component design is still on the drawing 
board. 

Now and in the interim before this ultimate is attained there 
is, of course, much need for retrofitting of the present plants so 
that they can produce to their optimum capacity and at the 
highest possible profits. However, such work is necessitated by 


deficiencies in our past knowledge and abilities in design. Such 
deficiencies must and should decrease as the overall design and 
prediction ability increases. 

The ultimate future success, therefore, depends upon our 
ability to produce mathematical models of the plants and of 
their related devices and processes. The ability to do this for 
time-varying requirements and for time-dependent upsets 
depends upon the knowledge of process dynamics. While 
traditionally the engineering of production plants has been on 
a ‘steady-state’ or ‘smooth-operation’ basis, allowing the use of 
algebraic equation models, there is now no serious question 
but that future design must be on a dynamic or unsteady-state 
basis and must therefore involve differential equations or 
related models. Again process dynamics knowledge makes this 
possible. 

It is therefore important to use this opportunity in order 
to review the recent progress in the field of process dynamics 
and its applications to industrial process control in the above 
light. This is an excellent occasion to establish the present status 
of the field, at least as viewed by this author, and to make some 
recommendations for future work which may serve to further 
the field and remedy some of the omissions noted at this 
time. 

It will be seen that future progress in process dynamics 
studies and the application of this knowledge to process control 
and process design problems depends on developments in three 
areas. These are: better methods of approximating complex 
responses by simple models; more complete and more exact 
characterizations of the non-linear parameters of process 
systems; and the development and use of faster and more 
capable computer systems for process simulation use. 

The reader’s attention is also invited to the several review 
articles and books covering the field of process dynamics 3 ’ 31 » 
88-u particularly the subject surveys of Archer and Rothfus 2 on 
distillation dynamics and control, Lapidus 20 on chemical reactor 
dynamics; and Williams and Morris 42 on heat exchanger 
responses. The report of the American Automatic Control 
Council edited by Berger 8 also contains much of value to the 
subject covered here. 
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Facts about Process Industry Systems 

In order to set the stage properly for a discussion of the 
mathematical modelling of process industry manufacturing 
systems, one should call attention to the important characteristics 
of such systems which distinguish them from those of other 
areas. Table 1 lists several such factors. They serve to emphasize 
a statement which is especially true for these industries: ‘The 
real world is non-linear, multivariable, and non-stochastic—all 
the things which make a theoretical analysis difficult 5 . 


The Uses of Process Dynamics in Process Plant Studies 


A Division of Objectives and of Approach 

At this time, process dynamics studies are being used for 
several purposes in process plant applications. However, these 
are such that they require only two main outlooks in developing 
the process dynamics concerned. These are: 

(1) Development of overall approximate models of process 
dynamics either from experimental work or from drastically 
lumped and/or linearized forms of theoretical plant mathematical 
models. The object here is to produce the simplest possible model 
which still adequately represents the phenomena studied or which 
provides the response required for a specific purpose. Some 
examples are: 

(a) Feedforward and feedback compensation in plant control 
systems, particularly analogue computer control systems 10 ’ 21 > 29 . 

(b) Linearized simulations for classroom demonstrations or 
control systems development work 29 . 

(2) Studies of plant design problems and simulations of plant 
start-up and major upset conditions. Here, in contrast to the 
above, the intention is to obtain as complete and as exact a 
duplication of plant behaviour as is possible. In fact, the limit 
of the study is usually the size of the analogue computing equip¬ 
ment available or the amount of digital computer calculation 
time which can be used 40 . 

These two outlooks have, of course, entirely different require¬ 
ments for simulation accuracy, variable excursion range, treat¬ 
ment of linearizations, etc. In addition as computers become 
ever more available and more capable, the disparity in approach 
used and results obtained from these two types of studies will 
necessarily increase. Both have a definite place. They are 
mentioned here only to call attention to the fact, mentioned 
later, that care must be taken in comparing the results of various 
studies to note the purpose for which they were developed. 

An excellent comparison of the amount of model complexity 
necessary for various applications of process dynamics is 
provided by the generalized chemical reactor model of Williams 
and Otto 43 and the several solutions proposed for it 7 ’ 10 ’ 12 > 13 > 17 

Eventually one must be able to develop and use dynamic 
models for complete plants of all types. As an example of such 
work, Borel has developed for this Congress his paper on the 
regulation of a hydro-electric plant. 


Future Gains from Process Dynamics Knowledge 
Design and Control Work 


in Process 


The gams which process dynamics knowledge can make to 
process control work are already well known from many sources. 
In addition, a knowledge of process dynamics offers the pos¬ 
sibility of making the following gains in plant design and opera- 


tion available to justify the time and expense necessary for its 
development: 

(1) In many plant situations the use of a continuously 
dynamic situation rather than a steady-state method of opera¬ 
tion will result in increased gains from a process such as in some 
batch reactions compared to continuous ones and pulsed ex¬ 
traction columns versus steady operation of the same columns. 
Process dynamics analyses can evaluate such gains and determine 
their economic worth. 

(2) Process dynamics knowledge allows the determination 
of the true nature of the operating extremes to be expected from 
a process under various conditions of upset. It can thus help to 
avoid excess costs arising from overdesign of the process and/or 
its control system. Likewise, it allows the determination of the 
amount of time off-specification following an upset and the 
equivalent amount of off-specification product formed. 

(3) Process dynamics information can supplement theoretical 
reasoning and steady-state investigations in elucidating the 
nature of some processes, thus helping to establish models useful 
for design as well as for control. 

The Formulation and Use of Theoretical Models 

It is well recognized that one can at present write theoretical 
mathematical models for practically all production processes 
to whatever degree of complexity one is capable of solving on the 
available computing equipment. That is, one already has a basic 
knowledge of the probable behaviour of processing systems and 
can list the parameters that will be important. However, this is 
only a minor part of our difficulties and there are still many 
problems requiring investigation. Some of these are: 

(1) One has at present only a rudimentary knowledge of the 
amount of detail in a mathematical model which is actually 
necessary to simulate the dynamic behaviour of a particular 
process to achieve a desired objective. This is particularly true 
in terms of approximate models and is due to several factors: 

(a) The tremendous range of complexity of the processes 
themselves as mentioned in Table 1. 

(b) The fact that there is no generally accepted definition of 
the goodness of fit required between experimental and theoretical 
or simulation data. 

(c) Since simulations can be used for many purposes, this 
in itself allows a tremendous breadth in the requirements for 
modelling detail and the resulting required closeness of fit be¬ 
tween theoretical and experimental data. 

(d) Deficiencies in literature examples of the use of theoretical 
process models. In many cases severe limitations in computing 
capacity available to investigators have put too restrictive limits 
on the modelling detail which was actually used. 

0) Process dynamics studies to date have tended to be 
generally either all experimental or all theoretical with little 
comparison of one type of data with the other for a particular 
piece of apparatus. 

(2) A great deal of work has been done recently on the 
linearization of the mathematical models of processes through 
the use of the technique of small perturbations. This has resulted 
in an ability to simulate much larger and more complex pieces of 
equipment and systems. However, this has itself raised several 
problems related to those just above. 
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Table L Vital Factors Concerning the Process Industries Important to 
a Discussion of Mathematical Modelling 

I These industries processes or separate operations are, as a rule, 
multistage, multiphase, and multicomponent and operate over 
wide ranges of temperature, pressure, and flow. In addition, they 
often must be treated as distributed parameter systems and usual¬ 
ly as non-linear ones. 

II A typical production plant will usually consist of from several to 
many such processes connected together to form a complete unit. 

III The so-called ‘time constants’ of the several important variables 
in a process may typically vary by one to several orders of magni¬ 
tude. This is particularly true when one is comparing temperature, 
composition and flow transients for the same process. 

IV As a general rule, these processes will be operated below capacity 
because of competition with other processes or even other com¬ 
pletely different products. Thus gains from optimizations must 
often be limited only to increased yields or lowered costs rather 
than being based on higher production levels from the plant. 

V Because of the rapidly moving pace of industrial research and 
the vagaries of the economy, management is usually anxious to 
rush a new plant into production as soon as possible after ap¬ 
proval of its construction. There is thus a supreme premium on 
time during the design and construction phases. 

VI The design and operating details between competing processes 
may vary slightly and still result in appreciable competitive ad¬ 
vantages to one process or the other. For this reason there is a 
reluctance to discuss actual operating process details or real re¬ 
search data in the literature. At the same time discovery of new 
but comparably small improvements in such operating methods 
or designs can be equally important. 

{a) What are the actual limitations on the linearization 
techniques used, i.e. through what ranges of the variables 
involved can the linearized simulation be perturbed and still 
duplicate the actual processes to some specified closeness of fit ? 

(b) Perhaps some process variables are more susceptible to 
linearization than others and a mixed linear-non-linear system 
is the proper answer. 

The above statements are particularly important when one 
is considering process control studies similar to Item (1) under 
the heading ‘Uses of Process Dynamics’ given above. These 
difficulties are-also of extreme importance as long as computing 
capacity limitations restrict the detail possible in a plant design 
or operation study. 

Experimental Methods for Determining Process Dynamics 

Activity in the experimental testing of process plants and 
equipment to determine the overall dynamics has been very 
active during the past few years. Chief interest has been con¬ 
centrated in the use of pulse testing methods 15 ’ 18 with subse¬ 
quent computer data reduction to frequency response or in the 
use of correlation methods. These latter have often used special 
computers 9 built specifically for this purpose. These automatic 
data-reduction techniques with computers have made it rela¬ 
tively easy to develop overall transfer function or performance 
function models relating a particular process output variable 
to a particular input variable change. This is especially true for 
pulse type inputs. Because of the special data reduction problems 
involved, however, higher-order effects may be lost or are at 
best very difficult to obtain. The following questions regarding 
these studies remain relatively unanswered as of this time: 


(1) What are the indispensable dependent variables of any 
particular process? That is, just how many separate transfer 
functions are necessary to describe the system adequately ? What 
are the best data-presentation methods for expressing this data? 

(2) Just as in the case of linearized small-perturbation type 
simulations what are the limits of input variable manipulation 
for which the linear transfer function representation is adequate? 

(3) What are the criteria to be followed in deciding whether 
higher-order effects may or may not be important in any specific 
case? 

Status of Process Dynamics Studies for Various 
Process Operations 

Studies of process dynamics work to date have been con¬ 
centrated mainly in three areas i heat exchangers, distillation 
columns, and chemical reactors. Heat exchangers have been very 
important because of their ready availability in nearly all uni¬ 
versities and the fact that they can be operated over wide ranges 
of operation with only steam and water, both of which are read¬ 
ily available in all laboratories. In addition, they provide in¬ 
triguing experimental models lor investigating the all-important 
problem of lumped parameter versus distributed parameter re¬ 
presentation 6 - 14 > 16 - 25 > 35 - 42 . Distillation columns are excellent 
examples of multivariable systems and lend themselves to many 
types of representation 1 - 2 - 4 > 5 > 19 > 21 - 21 - 29 > 32 > S3 ’ Also, they 
are extremely important commercially so that solutions to their 
responses and control problems are very valuable economically. 
Likewise, chemical reactors are very important commercially 
being the heart of any chemical process. They also are intriguing 
problems in process optimizations because of the complexities 
and the interactions taking place in chemical reactions and the 
presence of recycle streams in most cases 3 - 7 > 10 - 12 - 17> 29 ’ 28 ’ 

36 , 37 , 39 

Based on this one can make several statements on further 
work which should be carried out in these subjects. These fire 
given below: 

(1) Heat exchangers—A very large amount of work has been 
done in this field, so much so that additional work would have 
marginal value unless devoted to one of the following topics or 
others of related complexity: 

(a) Dynamics of fluid behaviour in the shell section of shell 
and tube exchangers and the effect of these flow changes upon 
tube side temperatures. 

(b) Dynamics of forced circulation and thermosiphon re¬ 
boilers, condensers, and other such heat transfer devices where 
a phase change in a flowing stream is of major importance. The 
paper of M’Pherson and Muscettola 25 treats this specific problem 
in the context of a nuclear reactor. The results will be valuable 
for ordinary boilers as well. 

(2) Distillation—While distillation has been a popular subject 
for study in the past, major problems exist in the following 
areas: 

(a) What are the valid approximate models which may be 
devised to represent the distillation system since the complete 
mathematical model of a ‘real’ column is completely unmanage¬ 
able by present methods? What are the limitations of each type 
of approximation that may be devised? In this regard, it should 
be noted that several of the papers of this Congress have concern- 
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ed themselves with the development of approximate models for 
distillation columns. I refer specifically to those of Izawa and 
Morinaga 19 , Moczek, Otto and Williams 24 , and Zavorka 44 . 

(b) Characterization of plate mixing and mass transfer effects 
necessary in the more exact simulations. In this Congress the 
vital nature of this subject is recognized by the reports of Ani¬ 
simov 1 who discusses plate mass transfer, and by Takamutsu 
and Nakanishi 34 who discuss both topics. 

(3) Reactors—The simpler reactor systems have been very 
extensively studied for the cases of relatively simple kinetic 
situations. Much work needs to be done yet on: 

(a) Complex reactor designs. (The paper of van der Heyden 
and van Nes 36 of this Congress discusses a very real problem in 
reactor design and control—the heat exchange system of a cata¬ 
lytic cracker 36 .) 

(b) Higher order and multiple reaction kinetic systems. (The 
paper of Volter 37 of this Congress treats this extremely complex 
and important subject.) 

(c) Optimization (dynamic and steady state) and adaptive 
control of real reactor situations. Reactor design and operation 
is the area where optimization and adaptation appears to hold 
the greatest promise. 

(4) Others Relatively little process dynamics and control 
work has been done for such chemical process operations as 
multiple effect evaporators, absorption columns, extraction 
vessels and columns, continuous filtration, drying and humidifi¬ 
cation, etc. Work is under way at several places on extraction 
which may well place this operation on a par with distillation in 
a year or so. While work has mainly been concentrated in the 
more common chemical and petroleum industry examples, it 
should be remembered that cement kilns, glass furnaces, blast 
furnaces, and smelters are all chemical reactors and the findings 
on the study of chemical reactors will eventually be translatable 
to these more complex cases. 

One place where much more work is necessary and where 
additional effort should be very rewarding is in the study and 
characterization of plastic flow, particularly in the extrusion of 
metals and plastics and the rolling and cold working of metals. 

General Notes Regarding Process Dynamics 

Some general observations regarding the problem areas in 
process dynamics can also be made at this time. These are: 

(1) Most difficulties in mathematical model-making resolve 
themselves directly to the characterization of fluid flow phenom¬ 
ena occurring in the process such as turbulence, liquid mixing, 
etc. In this respect the important work of Nichols 26 , and Olden- 
burger and Goodsorr 7 on the dynamics of hydraulic and pneu¬ 
matic control lines is extremely valuable. In addition, the work 
of Delvaux 14 and Takamutsu and Nakanishi 34 on fluid mixing 
in heat exchangers and on distillation column trays gives im¬ 
portant insights into the solutions to this problem. 

(2) Control in chemical processing almost always involves the 
manipulation of the flow of a fluid stream (here fluid may mean 
gas, liquid, or fluidized solid). 

Hence fluid flow problems are at the heart of all control 
investigations, while at the same time being the process dynamics 
phenomena which one least understands. 


The Place of the Computer in Process Dynamics Studies 

As mentioned earlier, the availability of computer capacity 
and speed of operation is one of the main deterrents to further 
progress in the application of process dynamics studies to plant 
operation and design problems. It will be helpful to establish the 
present status of such computer usage before proceeding. 

Analogues Versus Digitals for Process Simulations 

A point worth noting and commenting upon is the use of 
analogue versus digital computers for simulation problems, i. e. 
solution of ordinary and partial differential equations for the 
study of a transient process behaviour or of an automatic control 
response. The real criterion of applicability here is whether or 
not the computer can operate in ‘real time’. That is, can it com¬ 
pute sufficiently fast so that its answers appear faster or at least 
at the same rate at which the physical system being simulated 
would have produced them. Therefore, both the complexity and 
speed of response of the system being studied is important. In 
Figure 1 this is expressed as the order of the system (approxi¬ 
mately equal to the number of separate differential aquations 



Figure 1. An evaluation of digital computers as process simulators as a 
function of systems order and response speed (data are based on com¬ 
putation of distillation column dynamics) 


in the system) divided by the time constant of each cell (the 
time required for the cell output to respond 63-2 per cent to a 
step input change). Under such a criterion, Figure 1 shows the 
capability of the IBM 704 and IBM 7090 to compute the tran¬ 
sient response of a relatively complex, high-order, system, here 
a multicomponent distillation column. 

The indicated point is for a particular column studied 24 . In 
this case the IBM 704 was able to compute only 1 / 10 as fast as 
real time. In order to achieve column response speeds a reduction 
of system order, i. e. number of plates, to Vio of the original 
number or of the system complexity, i. e. detail of plate equi¬ 
librium and flow computations, by the same amount would be 
necessary. 
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Figure 2(a). Diagram of relative applicability of analogue and digital 
computers to process simulation problems 



System order 


Figure 2(b). Explanation of data in Figure 2(a) 


Figure 2(a) by pointing out the regions on the graph where each 
machine would be superior to the other for process simulations. 

It is truly unfortunate to note the number of chemical process 
systems which fall into the lower right-hand comer of Figure 2 (b). 
That is, they are too large for the analogue machine and too 
fast for the digital computers at present available. 

Computer Size and Convenience 

Despite the tremendous advances which have already been 
made in the use of computers in the process industries, it is still 
disappointing to the author to see the low order of complexity 
of the problems which can be handled with the computing 
facilities available to most of our process control and systems 
engineers. This is particularly so when one reflects upon the 
tremendously high order of the equations required to duplicate 
the action of even a relatively small and comparatively simple 
chemical plant. Thus there is no doubt that one will always be 
able to use still larger and faster computers as they become 
available up to the limit which particular companies can 
invest in such work. 

Reflection would lead one to judge that this limit could easily 
be as high as 0*5 per cent of the total invested capital of a pro¬ 
gressive and growing chemical or petroleum company not count¬ 
ing those computers used strictly for accounting purposes. It is 
hoped that equally large and capable computers can be provided 
for the universities. 

Necessary as the above growth in size and speed is, another 
factor is of even more importance for present and future growth. 
This is the question of accessibility of the machines to the average 
engineer. Methods must be developed to make it easy for any 
engineer who so desires to present his problem to the machine 
and to obtain his answers quickly and in a readily usable form. 
Unfortunately all computer work still requires the attention at 
some stage or other of the ‘expert 5 . The much touted ‘assembler 5 
and ‘compiler 5 routines have indeed made the task of program¬ 
ming much easier. However, even they require the assistance of 
an expert for their ‘debugging 5 . Also, only the expert computer 
man can get the most from the machine when memory space or 
computing time is in short supply even with the aid of the auto¬ 
matic programming aids available at present. Thus much work 
yet needs to be done on such computer software . 

Physical location is also a factor in computer use. In many 
cases a suitable computer problem is done by a less accurate 
method or not done at all because the computer was not con¬ 
veniently near to the engineer to use. 


Figure 2 shows the relationship of various sizes and types of 
analogue computer installations to the 704 and 7090 for simu¬ 
lation purposes. Analogue computers are limited directly in the 
system order which they can duplicate by the number of comput¬ 
ing elements they contain and indirectly by the accuracy of these 
elements. They are limited in speed of response by the type of 
multiplying gear used. Thus on Figure 2(a) , a particular ana¬ 
logue computer would be applicable to all combinations of order 
and of time constant above a horizontal line representing the 
type of multiplier used and to the left of a vertical line represent¬ 
ing the number of amplifiers it contains. The digital can ade¬ 
quately handle all combinations above the line representing it 
on the graph since it is limited theoretically only by its speed of 
computation or of information handling. Figure 2(b) exten s 


The Simulation of Fast, High-order Chemical Systems 

Thus, one of the very serious drawbacks in present day process 
control and process design practice is our inability to simulate 
large, fast chemical systems because present day analogue com¬ 
puters are too small and digital computers too slow. Unfortu¬ 
nately, also, available analogue computer accuracies prevent 
these machines becoming much larger than the largest of those, 
in use today (about 400 amplifiers) when applied to chemical 

problems. . . , - + 

However, there are several recent or imminent developments 
in analogue computer equipment and in operating techniques 
which promise to greatly aid and possibly even revolutionize 
analogue computer operation. Most of these involve digita 
techniques of one type or another. 
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One of the major present-day problems with analogue com¬ 
puter operation in the process industries is the difficulty of ade¬ 
quately representing a true time delay. The development of a 
suitable and reasonably priced magnetic tape or drum mechan¬ 
ism for this purpose would be a major boon to the field. Such 
devices are now under development. 

As the reader knows, analogue computers are especially de¬ 
signed for the solution of ordinary differential equations and do 
this extremely well while digital computers are very fast and 
accurate in the solution of algebraic equations but relatively slow 
with large systems of differential equations. In addition, ana¬ 
logues have only rudimentary logic capabilities while extremely 
complex logic operations are carried out directly on digital 
machines. There is thus a very fruitful potential in the joint use 
of these machines for the solution of large process problems such 
as being discussed here, which entail all three types of mathe¬ 
matical operation, solution of algebraic equations, and logical 
selection from several possible paths of action. Devices for con¬ 
necting the two computers together during operation are in use 
by several of the defense system contractors, such as Space 
Technology Laboratories, Convair Astronautics, and Inter¬ 
national Business Machines Company in the Unites States. 
Models of these devices are also under development for use by 
the process industries. 

Another area where aid is needed in analogue computation 
is in the simulation of distributed parameter systems and pro¬ 
cesses. Adequate mathematical representation of these systems 
requires the use of partial differential equations or extremely 
large sets of simultaneous ordinary differential equations. Thus 
their solution on analogue equipment requires a very large 
machine. However, the use of a modified form of the data storage 
drum mentioned above for true time delay simulation offers an 
aid in the solution of this type of problem. If preliminary results 
can be stored during the solution of a problem of this type on 
the analogue computer, it can be solved on a much smaller 
machine than required above by the use of a repetitive trial and 
error procedure which can be readily programmed. Again, these 
devices are under active development by the computer manu¬ 
facturers. Thus, digital devices promise to have a major effect 
on analogue computer operation in the process industries. 

Now consider a possible set of requirements for such a con¬ 
necting device as described above for use in solving industrial 
process simulation problems. 

(1) In order to handle reactor problems with their associated 
large number of composition values, the device must be able to 
store the instantaneous values of at least ten variables at each 
stage of the computations. These may be compositions, temper¬ 
atures, flows, etc. 

(2) For time delay representation purposes, it should be able 
to hold values of each variable for at least 2 min. 

(3) It should be able to give good resolution for frequencies 
as high as 10 c/sec. Thus sampling at a rate of at least 40 sam- 
ples/sec/variable would be necessary. 

(4) Combined with Item (2) above, this means that up to 4,800 
separate numerical values per variable must be stored for an 
adequate time delay representation. 

(5 l ( T 0mb i nati0n of Items (4 > and 0> requires that a storage 
capa i ty of at least 50,000 words be available for use with the 
analogue computer. 

(6) In order to allow adequate communication between the 


two machines (analogue and digital) at least ten channels each 
way will be necessary. Multiplexed analogue-digital and digital- 
analogue converters may be used if their speed is adequate for 
the task outlined above. 

With such a combined analogue-digital computer the analogue 
computer could perform all integrations and such operations 
which are slow on the digital machine while the latter performed 
the arithmetic operations and function generation duties which 
require large amounts of equipment on the analogue. Tn this way 
a large inroad into the lower right-hand area of Figure 2(b) 
could be possible. 
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Determination of System Dynamics by Use of Adjustable Models 

E. BLANDHOL and J. G. BALCHEN 


Summary 

A method for determining system dynamics by using a simulated 
model with adjustable parameters is described, and practical problems 
dealt with. Application of the statistical computer ISAC greatly sim¬ 
plifies the experimental work. The usefulness of the method is illus¬ 
trated by some experimental results. A second-order system containing 
four parameters, including a pure time delay, was easily identified, 
with all parameter errors less than 3 per cent. 

Sommaire 

Une methode de determination de la dynamique d’un systeme au 
moyen d une simulation a l’aide d’un modele a parametres reglables, 
est decrite et certains aspects pratiques de l’application de cette methode 
sont indiques. 

L utilisation du calculateur statistique ISAC simplifie beaucoup le 
travail experimental. L’utilite de cette methode est mise en evidence par 
un certain nombre de resultats experimentaux. Un systeme du 2e ordre, 
avec quatre parametres dont un retard pur, a ete facilement identifie, 

1 erreur de la determination de chacun des parametres etant inferieure 
a 3 pour cent. 

Zusammenfassung 

Der Aufsatz beschreibt ein Verfahren zur Ermittlung des dynamischen 
Verhaltens von Systemen durch Verwendung eines nachgebildeten 
Modells mit emstellbaren Parametern und betrachtet praktische Pro- 
b eme. Die Benutzung des statistischen Rechners ISAC vereinfacht den 
Untersuchungsaufwand betrachtlich. Einige Versuchsergebnisse zeigen 
den Nutzen der Methode. Bei einem System zweiter Ordnung, das vier 
Parameter und eine reine Totzeit enthalt, konnten auf einfache Weise 
alle Parameter mit einem Fehler, der kleiner als 3% ist, bestimmt 


Principle of the Model Method 

The general principle of what, in this paper, is called the 
‘model method’, is illustrated in Figure L 

We want to estimate the dynamics of some physical system g 
by applying a test signal x (/) during some finite time T. The 
system output contains additive noise n (/), so that only 
z (0 = y (0 + n 00 is measurable. Now a model h of the system 
is simulated, and the input x ( t ) is applied to the system and the 
model in parallel. The model usually has a fixed structure, but 
with several adjustable parameters. The model output u if) is 
then compared with the noisy system output z (t) according to 
some error criterion, and the model parameters are adjusted 
until the error becomes a minimum, following some pre¬ 
selected strategy. The resulting model configuration is said to 
be the ‘optimum’ approximation to the system. 



l_j 

Figure 1 . Principle of the model method 


Introduction 

In recent years, the method of determining the dynamic prop- 
erties of physical systems by using a simulated model with 
adjustable parameters has been given considerable attention 
Much work has also been devoted to the development of more, 
or less, ingenious automatic adjustment procedures, so that the 

method could, for instance, be included in an adaptive control 
system. 

Most of the rather few reported experiments have employed 
quite complicated and expensive equipment both for the model 
and the adjusting mechanism 1 - 5 . All that is needed in many 
cases, however, is some relatively simple equipment used in 
connection with a model which is manually adjusted to optimum. 
The method may then be a very powerful one for studying the 

v y “ ° f mdUSt / ial pr0Cesses and oth er non-adaptive 
systems. This paper shows that the statistical computer ISAC 6 . 7 
m connection with an analogue model, is well suited for this 

report® 6 ' A m ° re detalled account is found in the technical 


The parameter adjustments must be based on the measured 
error. In automatic model adjustment schemes an ‘adjustment 
computer’ is inserted as a feedback link, as indicated in Figure I. 
In simpler cases the computer may conveniently be replaced by 
a human operator; this is the principle employed in our 
experiments. 

Practical Problems 

Efficient use of the model method requires careful con- 
* 1,1100 of a number of practical problems, some of which 
are mentioned below. 

Choice of Model Structure 

™.? 0rtant <l uestion when using the model 
method is. What should the model structure be?’ 

in J he a P ff 0CeSS ldentificati on problem may basically be posed 
in two different ways: p 

with 1 !,^ 118 3 PhySkal SySt6m Wh0Se Structure is known, but 

reoSired^r When these P ara ™ter values are 

required, the model structure should, of course, be chosen equal 
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to the system structure. If some simplification of the model is 
necessary, the kind of structural approximation made is always 
known. 

(2) In other cases the system may be a ‘black box’ with a 
more or less unknown structure. In this event the system dynamics 
are merely approximated by some mathematical model of 
specified structure. A model transfer function with two poles, 
one zero, and a pure time delay may, for instance, be specified. 
A model satisfying these requirements may be synthesized in 
many ways and the resulting model structure may be quite 
different from the true system structure, even if the order of the 
model transfer function has been correctly chosen. On the other 
hand, it was experimentally observed that the type of model 
structure has a great influence on the accuracy and speed of 
convergence of the method. The choice of model structure 
should therefore be judged from this point of view. The model 
should also be chosen according to the kind of information 
desired. Thus, if the impulse response is required, a time domain 
model should be used, which may give an excellent approxima¬ 
tion to g (/), although its transfer function may be quite different 
from that of the system. 


Time Sealing 

Many systems, for instance industrial processes, have such 
large time constants that a model cannot be simulated on an 
analogue computer with reasonable component values. More¬ 
over, experimental optimization of the model in real time would 
take a prohibitively long time. Some kind of time scaling is 
then necessary. 


Error Criterion 

A number of different error criteria have been proposed for 
use in model adjustments. Denoting the error by <2 (t): 

e(t) = z(t)—u(t) (1) 


the criterion 
square error 


used 


in our experiments was the integrated 


2 (t)dt 


( 2 ) 


which yields least-squares estimates for the model parameters . 
Moreover, when the model output u(t) is a linear function ot 
the model parameters, parabolic minima are obtained, which 
is of great practical importance. 


Initial Condit ions 

When model adjustments are based on a finite record which 
is cyclically repeated, as in our experiments, the problem of what 
initial conditions to apply to the model in each run occurs, 
possible, the measurements should be made with zero system 
initial conditions. Sometimes, for instance when measuring 
during normal system operation, this may e impossi e. > 
if the initial conditions cannot be neglected or evalua 
separately, they must be included in the model as additio 
adjustable parameters. 


values. The adjustment strategy should lead uniquely to this 
absolute minimum when starting from arbitrary initial values. 
Possible sources of difficulty are: (#) existence of local minima, 
(b) poor sensitivity of e to some parameters, (c) poor convergence 
for certain model structures and strategies, and (d) lack of 
orthogonality, i.e. dependence of optimum value of one para¬ 
meter on other parameter values. 

When using an automatic ‘adjustment computer’ the strategy 
may involve quite complicated mathematical calculations. 
Examples of such strategies are the method of steepest descent 3 » 
and the gradient method used in Feldbaum’s optimalizer 2 ’ 10 . 

For manual optimization another kind of strategy is desirable, 
avoiding mathematical calculations or simultaneous adjustment 
of several parameters. One obvious possibility, used with success 
in our experiments, and also suggested by Levin 9 , is a 
cyclical adjustment of the parameters, one at a time, until local 
minima occur. Using ISAC, the error measure e is plotted on 
an xy recorder. The parameter adjustments are then based quite 
simply on visual inspection of the plotted s points. 

If the error e ( t ) is a linear function of the adjustable para¬ 
meters oc l9 oc 2 , ... oc n , the error measure e becomes, in matrix 
notation, a quadratic form. Then the cyclical adjustment strategy 
is equivalent to the Gauss-Seidel iteration procedure for solving 
the set of linear condition equations for a minimum in s: 


0a 

0a f 


= 0 (i = 1,2,..., n) 


(3) 


Since, from the definition (2), & is positive definite, it has a 
unique parabolic minimum and the adjustment procedure is 
known to converge. When a is a non-linear function of the 
parameters the equation set (3) will still be approximately 
linear in the vicinity of optimum. 

When s is a quadratic form it may, in two dimensions, be 
represented by a family of ellipses e — const. The approach to 
optimum will then, with cyclical parameter adjustments, look 
as in Figure 2. The adjustment of one parameter will proceed 
until the trajectory becomes tangent to the family of ellipses. The 
loci of such points will be two diameters in the ellipses, as 
indicated in the figure. The speed of convergence clearly depends 
on the angle (j) between these diameters. In Figure 2 (a) the 
convergence is good, whereas in (b), where the performance 
surface has the form of a narrow ‘valley’, the convergence is very 
poor. The shape and orientation of the ellipses and the angle (p 
may be expressed in terms of the elements in the coefficient 
matrix R for e. 8 These matrix elements depend m a rather 
complicated manner on the input signal, the system function, 
the output noise and the integration time. 

In Figure 2 the parameter axes do not coincide with the 
semi-axes of the ellipses. When they do, the parameter adjust- 
ments become orthogonal. In that case, the relative sensitivity 
of £ to the various model parameters is directly related to the 
corresponding eigenvalues in the coefficient matrix . e 
optimum values of those model parameters^ corresponding; to 
the smallest eigenvalues will be most sensitive to noisejhe 
analytical requirement for orthogonal adjustments is that R 
be a diagonal matrix. 


Adjustment Strategy 

For « adjustable parameters the error crite ^ n 6 rep ^® 
a surface in (« -|- I )-dimensional space, with an absohfie 
minimum corresponding to the optimum set of parameter 


Use of the Statistical Computer ISAC in Model Experiments 

ic a re, 7 j s a self-contained analogue computer, primarily 
intended for automatic computation of correlation functions. 
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power spectra or amplitude distribution functions. It consists 
of a special-purpose magnetic tape recorder, electronic com¬ 
puting units and an xy recorder. One to three input signals are 
simultaneously recorded on a magnetic tape loop. There are 
eight recording tape speeds, covering a range of 1:1024. Playback 
is always performed at full tape speed. Maximum effective 
recording time is 68 min, playback revolution time 5 sec Input 
frequency range is 0-200 c/sec at full tape speed, being propor¬ 
tional to tape speed. Two multi-track magnetic heads are used, 
one (A) fixed, the other (B) movable along a slide to obtain the 
time displacement r during correlation. The results are plotted 
on the xy recorder, and the automatic computation is controlled 
by relay circuits, triggered from two small holes in the tape loop *. 

The use of ISAC in model experiments is illustrated in 
Figure 3. The input x (/) and the output y (t) of the system to be 
studied are recorded on a tape loop, and then played back 
continuously during the adjustment procedure. The playback 
signal Xj, corresponding to the system input, is applied to the 
model, simulated on an analogue computer. The other playback 
signal y x is subtracted from the model output to yield e(t), which 
is then squared and integrated. By using the integrator in ISAC 

* ISAC is now commercially available. 


Figure 2. Approach to optimum with cyclical adjustment of parameters , 
for two different shapes of the e surface 



the resulting value of £ may be automatically plotted, one point 
per revolution of the loop. Operation and resetting of the model 
is controlled by the triggering circuits in ISAC, so that identical 
initial conditions are maintained in all runs. 

The time-scaling problem is easily solved by recording the 
test signals at a sufficiently low tape speed. The model is then 
simulated in playback time scale, where reasonable component 
values may be used. 

The inclusion of a pure time delay in a model is usually a 
difficult problem. In ISAC this may easily be achieved by simply 
displacing head B during playback, yielding a delay 0 < r < 320 
msec in playback time scale, or up to 0 < r < 320 sec in real time. 

Experimental Results 

The applicability and versatility of the model method were 
studied by making quite an extensive series of experiments, 
covering several combinations of system structures, model 
structures, input signals, etc. For checking purposes ail measure¬ 
ments were made on known systems, simulated on the analogue 
computer with zero initial conditions. One first-order and one 
second-order system were investigated, with or without a pure 
time delay included. 



Real time j Machine time 


Figure 3. Use of ISAC in model experiments 
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The total number of adjustable parameters must be kept 
rather small if the adjustment procedure is to converge with 
reasonable speed and accuracy. On the other hand, it is known 
that the impulse response of many higher-order systems may be 
fairly well approximated by the response of a system with two 
time constants and a pure time delay. Such a system, with one 
gain factor, two time constants and a time delay, or a total of 
four parameters, was therefore given special attention. 

In all measurements the system input x, the output y and 
the noisy output z were recorded on I SAC, so that results 
with and without noise could be directly compared for the same 
test signal. All signals were also recorded on a Brush recorder. 
Some representative results are given below. 


First-order System 

Referred to real (recording) time scale the system transfer 
function G x {s) was chosen as 


b 0 + biS 


e ToS= i—;— e ' 
1 + s 


where t 0 = 6-11 sec corresponds to 30 mm displacement of 
head B. The corresponding model transfer function H 1 (s) is 
written 




An input signal x(t) was used with Gaussian noise of bandwidth 
10 rad/sec. The playback-recording time-scaling factor p was 64, 
giving a recording time of 256 sec. The output noise n (t) was 
also Gaussian of bandwidth 10 rad/sec, statistically independent 
of the input x (t). Recordings of the three signals x, 4 y and 4 z 
are shown in Figure 4. 

In the model transfer function (5) ($ 0 or must be normalized 
to 1. It was experimentally observed that in order to obtain 
useful results one must always use = 1. The reason is that 
for most input signals, e will be more sensitive to errors in gain 
than in the time constant. Thus, when the gain depends on two 
parameters, <x 0 and /9 0 , and adjustments are started from arbitrary 
initial values, one will, after only one or two adjustment cycles, 
end up with a ratio a 0 /jff 0 close to the theoretical gain, and a 
fairly low value of e. But then no further reduction in a can be 
observed, although the absolute values of a 0 , /3 0 and /? x may all 
be completely wrong. This also applies to higher-order models. 

The playback circuit diagram is shown in Figure 5. The 
d.c. values of the two IS AC demodulator outputs are suppressed, 
and some extra amplifiers are added to obtain the necessary 
amplification and uniform loading of all parameter potentio¬ 
meters. 

For the signals in Figure 4 the model was optimized with 
and without output noise. Starting with the arbitrary initial 
values <x 0 = 0*6, p 1 = 0-714, r = 20 mm, the following optimum 
values with noise were found after a total of 10 parameter 



Figure 4 . Test signals for first-order system 



Figure 5 . Circuit diagram for first-order transfer function model 


605 






E. BLANDHOL AND J. G. BALCHEN 



Figure 6. Optimum e curves for first-order model without noise 


adjustments (finding four local minima for oc 0 and fi l9 and tw 
local minima for x): 

a o = 0*980 Pi = 1-020 t = 30-0 mm = 0-081 (6 

Here R is a figure of merit, defined as 




4m (0 dt 


z 2 (t)dt 


where « mln is the minimum value of s obtained with the optimur 
model, and £, is the value of s when the model is identically zero 
Note that e s is easily obtained experimentally by simply breakin 
the connection from pot. (16) to amplifier (7) in Figure 5. 

When the adjustments were started with « 0 or ft the com 
pletely ‘wild’ values « 0 = 0 or ft = 4-17 were obtained in th 
rs run, with only a slight decrease in e. x at once converge! 
to the value 30-25 mm, decreasing e by a factor of 8, in spite o 
the initial errors in a 0 and ft. The reason for this is that th. 
initial errorm t is so large that the outputs y and u are practical!' 
uncorrelated, no matter what values we give « 0 and ft. Thi 
important conclusion is that when studying a system where timf 

s£r n rr occ t ur ’. themodei ^ be *** *»■ T at an ^ 

stage of the optimization process. 


Starting with the same initial values without noise, 10 para¬ 
meter adjustments lead to the optimum values: 

a 0 = 0-960 ft =0-994 t = 30-0 mm R = 0-0012 (8) 

The plotted optimum e curves are shown in Figure 6. The curves 
for oc 0 and 1/ft are seen to be approximately parabolic. The 
r curve (note different ordinate scale) becomes flat for large 

errors in r, when y and u become uncorrelated, as mentioned 
above. 

To obtain the high sensitivity used in the curves for a 0 and ft 
extreme accuracy is needed in the experimental equipment. All 
e points in the three curves were plotted three times, but the 
spread is hardly noticeable. Smooth curves may also easily be 
drawn through all computed points. Both checks verify the high 
accuracy and stability existent in the computer ISAC. 

The values found in (6) and (8) are all within 4 per cent of 
their theoretical values. The largest error occurs in a 0 , but 
further tests suggested the presence of a systematic error 
always giving oc 0 < 1. 

A recording time of 256 sec is quite large in relation to the 
system time constant and the input noise bandwidth. In this 
case evaluation of the model parameters from statistical correla¬ 
tion functions, also computed on ISAC, gave about 3 per cent 
error in gain and 16 per cent error in time constant 8 . 
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To test the accuracy of the model method for short recording 
times, when statistical techniques are quite impossible, a record¬ 
ing time of only 8 sec was tried, with all other test conditions 
unchanged, except that no time delay was used. The following 
quite satisfactory results were obtained, with rapid convergence: 

« 0 Pi R 

Without noise 0-984 1-025 0-0007 (9) 

With noise 1-110 1-099 0-065 


Second-order System 

in recording time scale, 


0 2 ( 5 ) = ^ + -^e- 

= 1 e -T0S__1_ e "*OS MQ\ 

l+14s + 04s 2 (l + s)(lT-04s) 

where t () - - 30-5 mm. The same structure was used in the model, 
with /? 0 normalized to 1. The appropriate modification of the 
circuit diagram in Figure 5 is shown in Figure 7; x (t) and n ( t ) 
were maintained as for the first-order system. 

First a 256 sec recording was studied. As arbitrary initial 
parameter values were chosen oc 0 = 0-6, /? 2 — 0*286, ^ = 2*4, 
t = 20 mm, The convergence was rapid, and without noise the 
following values were obtained after 18 parameter adjustments: 


oc 0 = 1-002 /?! = 1440 
= 0-394 t = 30-55 


R = 0-0008 


( 11 ) 


As for the first-order system y and u were found to be uncorre¬ 
lated for r = 20 mm. r thus had to be adjusted first, at once 


leading to r = 30*75 mm. The plotted optimum s curves are 
shown in Figure 8. Due to the high model sensitivity a slight 
spread in the triple-plotted points is observed. 

The convergence was satisfactory with noise also, yielding 
the final result: 


a 0 = 1*014 ^ = 1440 
j5 2 = 0417 t = 30-55 


( 12 ) 


The optimum parameter values found in (11) and (12) are 
seen to be very accurate, the maximum error being 2*9 per cent 
without noise and 4*3 per cent with noise. 

When trying to evaluate the second-order system by correla¬ 
tion techniques, the cross correlation function must be approxi¬ 
mated by three exponential terms 8 . Due to the highly non- 
orthogonal nature of the exponential functions this would 
require an excessive data accuracy which by no means is 
attainable in experimental correlation functions, and was thus 
not attempted. 

Finally, an 8 sec recording was studied, with no time delay 
in the system. Even then the convergence and accuracy were 
quite satisfactory both with and without noise. The following 
results were obtained: 

a 0 /?i /? 2 R 

Without noise 0-962 1-400 0-396 0-002 

(13) 

With noise 0-940 1-400 0-357 0-34 


Parameter Determination in Multi-input and Non-linear Systems 

In the previous sections the noise was regarded as being 
added to the system output and being non-measurable. Actually, 
in many practical cases, what has here been considered as noise 



Figure 7. Modification of Figure 5 for second-order transfer function model 



I ™rZj j r ‘ i ‘ 1 ■ , 1 

Figure 8. Optimum e curves for second-order model without noise 
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may well be the response of another measurable input to the 
system. In such cases it is certainly advantageous to record this 
input (x 2 ) as well as the ordinary input (jq) considered before. 
Now a. model having two channels (I and II, from jq and x 2 
respectively) can be made. The model adjustment procedure 
could be as follows: first the channel I from jq to u is adjusted 
to^ optimum, regarding x 2 as noise. This channel is then main¬ 
tained while adjusting channel II till optimum is reached. 
Maintaining channel II, a new adjustment of channel I can be 
made leading to an almost complete compensation of the effect 
of the. ‘noise’ input x 2 . The computer ISAC, having three 
recording channels, is well suited for such purposes as well. 

Identification of non-linear systems has received very little 
attention in the literature so far. Model adjustment procedures 
as outlined here appear to be quite promising in this respect 
as well. If the system structure is known, say by its differential 
equation, and some non-linearity appears in it which can be 
characterized by a small number of parameters, then the problem 
of adjusting a non-linear model is essentially the same as in the 
linear case. Certainly the magnitude of the signals now becomes 
more important than in the linear case. The success in character¬ 
izing a non-linear element is thus dependent upon having signals 
that actually reveal the non-linear phenomena. 

Even though only introductory investigations have been 
made so far on these latter problems quite interesting results 
seem to be within reach. 

Conclusions 

It has been demonstrated that the computer ISAC is extreme¬ 
ly well suited for model experiments, providing recording and 
cyclical playback of the test signals, with a time scaling factor 
up to 1,024, and easily permitting the inclusion of a time delay 
in the model. The results obtained for the practically important 
second-order model, containing four adjustable parameters, 
including a pure time delay, give a very promising illustration 
of the potentialities of the model method, when precision 
equipment and correct experimental techniques are used. 


The material presented here is based upon the results of a re¬ 
search project at the Division of Automatic Control , the Technical 
University of Norway, sponsored by the Royal Norwegian Council 
foi Scientific and Industrial Research. The support is highly 
acknowledged. 
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DISCUSSION 


A. Hakala, Kymmene Aktiebolag, Kuusankoski, Finland 

This method and device could be a useful tool for determining the 
dynamics of processes during their normal operation. 

Have the authors made any actual tests in the field, and if so what 
is their idea about the capabilities of this method and equipment ? 

K. S. P. Kumar, C.I.S.L. School of Electrical Engineering, Purdue 
University, Lafayette , Indiana, U.S.A. 

(1) The modelling methods can be viewed as multi-point boundary 
value problems and very effective computational results obtained via 
the method of quasi-linearization 1 . Suppose the control system to 
be given by 

x=f(x,u, P> t) ( 1 ) 

where u is the control vector, x is the state vector and p are some 
unknown constant parameters of the system like, for example, time 
constants, gains, etc. As p are constants, they are written as 

P = 0 ( 2 ) 


Measurements are made on the control vector and state vector. It is 
unnecessary to have access to the entire state vector. It is enough if 
only a few components of it, or linear combinations thereof, are 
measured. Now eqns (1) and (2) are solved with the given boundary 
conditions on x. The method of quasi-linearization remains exactly 
the same even if eqn (1) is non-linear. 

Both cases, known structure and unknown parameters and un¬ 
known structure and unknown parameters, have been examined by 
the above method at the Control and Information Systems Laboratory 
at Purdue University with excellent results. These are under preparation 
for publication. Evidence shows that this method has great promise 
towards on-line control. 

(2) Did the authors try any steepest descent methods and if so how 
did they work out ? 


Reference 

1 Kalaba, R. On non-linear differential equations, the maximum 
operator, and monotone convergence* J. Math. Mech. 8 (1959) 
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A. J. Niemi, University of Oulu , Hoikantie 14 A 7, Oulu , Finland 

The impulse response of a system is simply the inverse transform of its 
transfer function and hence contains the same parameters. With 
respect to an impulse input, the frequency domain model, i.e. the 
transfer function, and the time domain model are therefore essentially 
equivalent. 

However, the authors recommend the use of a time domain model 
when looking for the impulse response of a system. I would like to 
know the reason for this preference and the real advantages gained 
by using the time domain model instead of a transfer function in 
this case. 


H. Gorecki, Academy of Mining and Metallurgy, Gramatyka 7\5, 

Krakow , Poland 

What is the effect of multiplicative noise in the input signal, when due 
to changes of system parameters, for instance ? 

Has there been any research into the identification of the system 
structure? For example, at the Chair of Automatic Control in the 
Academy of Mining and Metallurgy in Krakow we determine the 
structure of non-linear control systems or processes by the frequency 
response method. Suppose it is required to establish the equation of 
the plant: 

x'+a 2 x+f(x)x+a 0 x=0 ( 1 ) 

or 

x'+^^ + ^ + gC*)^ u) 

For these equations the frequency characteristics are determined 
as families of curves for different input signal amplitudes. In the case 
of eqn (1) the distance between the curves of the real part is pro¬ 
portional to this amplitude, whereas in the case of eqn (2) this is true 
for the curves of the imaginary part. The method might be extended 
to higher-order systems. 


R. W. H. Sargent, Chemical Engineering Dept., Imperial College, 
London S . W. 7 

May I ask if the authors have made any experiments to test the con¬ 
vergence of their proposed method of alternate adjustment for multi¬ 
input systems? 

It is easy to construct response surfaces for which the cyclical 
adjustment of variables will not converge on the optimum (even a 
local one), and one would intuitively expect the same type of behaviour 
in this multi-input problem. 


R.j.A. Paul, College of Aeronautics, Cranfield, Bletchley, Bucks, 
England 

(1) First- and second-order systems: Since the parameter adjustments 
in the model are not orthogonal, how many iterations were necessary 
to achieve the optimum parameters? The necessity for finding the 
value of the transportation lag, as a first requirement, appears to be 


valid. 

(2) Higher-order systems: Have the authors carried out any ex¬ 
perimental investigations with high-order systems approximated with 
the simplified model comprising a transportation lag, a zero and a 
pair of conjugate complex poles? If so, were any indications evident 
regarding the limitations of this approach? 

Investigations 1 indicate that if the poles of the high-order system 
are close together then it is extremely difficult to obtain a reasonable 
approximation with a simplified model. 


Reference 

i Paul, R. J.A. Determination of dynamical models for adapt^e 
control systems. College of Aeronautics, Note No. 128 {.Wt>2) 


C. H. P. Brookes, Department of Engineering Science , Oxford Uni¬ 
versity, Oxford , England 

At Oxford some work has been done using methods similar to those 
of Messrs. Blandhol and Balchen on models which are manually ad¬ 
justed and optimized. The approach has been to investigate the 
accuracy required in a model in order for it to be able to predict either 
system performance or an optimum control setting. No attempt is 
made to discover the position of the system’s poles or its gain; it is 
considered sufficient that the model is able to represent the system 
adequately. 

Type 1 systems with unity feedback have been considered up to 
date with a simple lag compensation network of form (A s +1 )/(Bs +1) 
included in the forward path. The model was set up in the same way 
with similar compensation. It consisted of a Type 1 servo with variable 
gain and pole positions. 

Step inputs were applied to both system and model and the model 
parameters were adjusted to minimize a function of the system-model 
error, usually the ITAE or ISE. Systems of up to tenth order, with and 
without complex open-loop poles, were simulated on an analogue 
computer and for each system several models of different complexity 
were used. Models were judged on their ability to predict the compen¬ 
sation which minimized the system ITAE and their prediction of the 
system output trajectory after a change in the compensation networks. 

One interesting conclusion from this work is that if the model 
consists of poles confined to the real axis of the s plane then the best 
configuration for a Type 1 model used to predict higher-order systems 
has a pole at s = 0 and a multiple pole somewhere along the real axis. 
This reduces the number of parameters to be adjusted in the modelling 
process to two, the gain and multiple pole position. The problem of 
modelling is eased considerably, especially when it is necessary to use 
models above third order. 

The result applied for all the systems considered so far. The two 
variables may be orthogonalized easily by using standard techniques, 
but it has been observed experimentally that if the adjustments are 
made on the velocity constant and multiple pole position the settings 
are nearly independent. Model performance, of course, deteriorates 
as the difference in order between system and model grows, but for 
models above fourth order the deterioration is much less marked. The 
use of a second-order model in a set-up as described above yields very 
poor prediction even for fourth-order systems. 

Random input signals and more complicated compensation net¬ 
works have been tried and the results regarding model configuration 
are the same. It is envisaged that a model of a very complex system 
will consist of several of these multi-pole models linked together. 


H. J. MEmmow, Pembroke College, Cambridge University, Cambridge, 
England 

Manual adjustment of the model parameters requires that their 
setting be non-interacting. For a model whose output is a linear 
function of the variable parameters, i.e. 


m(0= E a i $»(0 


i= 1 


a simple self-adjusting procedure has been developed 1 which does not 
require that the setting of on be non-interacting. Instead of determining 
the minimum value of 


e 2 ( t ) dt 


by successive adjustments to the parameters oh , the proposed self- 
adjusting procedure solves the simultaneous equations. 


lf T 

Tjo 


e(t)^i(t)dt=0 i=1,2,..., 


N 


by continuously adjusting the «. The setting time for the identification 
procedure may be shortened at the expense of accuracy of the para- 
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meter settings, which corresponds to using shorter interval lengths of 
the input a(/). A repetitive test record of the system input is not 
required, and the identification may be carried out continuously using 
the normal system input. 

Where the model output is not a linear function of the model 
parameters, as is given in the examples with variable poles in the model 
transfer function 


H l (s) = 


Po + filS 


•e 


the equations for setting the model parameters automatically may be 
unstable outside the small lines region near the stable equilibrium 
point. There may also exist unstable equilibrium points. 


Reference 

1 Meyerhoff, H. J. Self adjusting process models. Thesis, Cambridge 
University (1963) 


J. Rakowski, Institute of Power, Woloska 88 m 53, Warsaw 12, Poland 

The examples in the paper show how the model method can be used 
to determine the dynamics of one-input and one-output systems. 

The following question therefore arises: how could this method 
be used to determine the dynamic properties of a multi-input and 
multi-output system. For example, in the case of a steam generator 
one has to consider at least three main input and three output signals, 
forming three main control loops, i.e. three separate controllers. The 
structure of this system is complicated. There could be also some 
difficulty with opening (disconnecting) more than one of the control 
loops during the period of the test. 

Can the described model method be applied to determine the 
dynamics of such a system? How should one attack this problem? 


K. Reinisch, Institut fur Regelungstechnik, Heidefliigel 3, Dresden, 
Germany 

Applying deterministic signals (step functions and similar types), in¬ 
stead of stochastic ones, to the parallel compensation method in¬ 
vestigated by the authors, one may use another adjustment strategy 1 . 
This gives the time constants (poles and zeros) of the plant by the first 
approximation step with an accuracy of about 5-10 per cent and 
therefore shortens the identification process. For a class of plants this 
method allows the structure of the plant transfer function to be 
identified. It would be interesting to know whether the authors did 
some work in this direction, too. 

Reference 

1 Reinisch, K. Verwendung eines Modellregelkreises zur Gewinnung 
einfacher Bemessungsregeln fur Regelkreise und zur Ermittlung 
der Kennwerte von Regelstrecken. Zmsr 5, H. 6 (1962), 245-251 


A. A. Feldbaum, Institute of Automatics and Telemechanics, Kalan - 

tchevskaja 15a, Moscow, U.S.S.R. 

The paper is concerned with an extremely topical question and clearly 
demonstrates the convenience of determining the dynamic character¬ 
istics of plants with the aid of the ISAC computer. As regards the 
content of the paper, I make the following comments of a basically 
theoretical and also practical nature. 

(1) What is the advantage of using an adaptive analogue for 
determining the dynamic characteristics of plants, as compared with 
other known methods? For idealized problems it is possible to find 
the optimal algorithm for processing the information coming from 
the plant input and output; this information is mixed with random 
noise. 


It can be shown that the structural scheme which corresponds with 
the optimal algorithm does not match that examined in the paper. 
Hence the scheme of the adaptive analogue is not, from a theoretical 
viewpoint, the best one. Of course, building the scheme in Figure 1 of 
the paper does not require the solution of any complex theoretical 
problems and this is its practical advantage. 

But does not this scheme have any fundamental advantages ? It 
would be interesting to know the authors’ view of this. 

(2) The paper examines a process of optimization which is effected 
manually. However, experience in the Soviet Union indicates that 
man copes badly with the complex problems of optimization (when 
there is a comparatively large number of variables, when the extern¬ 
alized function has complex form, when there is a large number of 
trials with various initial values, different variants of analogue struc¬ 
ture, etc.). The automatic optimizers used in the Soviet Union in 
equipment for automatic synthesis of optimal systems are compara¬ 
tively simple instruments, but the problems solved by this equipment 
are not within the power of man, who would attempt to solve them 
manually. 

When using the method of two analogues connected in parallel, 
in which the parameter values differ by one trial step (as in Dr. Mesch’s 
paper), an even simpler instrument, of the controller type, can be used. 

P. M. E. M. van der Grinten, Central Laboratory, Staatsmijnen, 

Geleen, Netherlands 

In your paper, as in many others on model updating, not much atten¬ 
tion is given to the requirements imposed on the test signal x(t). Yet 
the influence of the properties of the test signals (power spectrum and 
possibly cross-power spectrum), on the speed and accuracy of con¬ 
vergence, is presumably as equally important as the choice of the 
model structure. Your statement that an apparently good approxi¬ 
mation. of the impulse response sometimes corresponds to a poor 
approximation of the transfer function, will perhaps also find its 
explanation m the application of test signals of a rather narrow 
bandwidth. 


H. A. Barker, Glasgow University, Glasgow ) W. 2, Scotland 

The method of model optimization described here appears to offer no 
advantages over methods in which the integral error-squared signal is 
used to adjust the model parameters directly by means of a simple 
feedback loop. These methods are applicable to linear and non-linear 
multivariable systems, and may be used for manual or automatic 
adjustment when the effects of model parameter adjustments have 
been made independent by an orthogonalization procedure, or for 
simultaneous automatic adjustment when these effects are not neces¬ 
sarily independent. The optimization time is less than that required 
for the models described in this paper, and it is not necessarily in¬ 
creased if the number of model parameters is increased. A single run of 
the system signal records is sufficient for the optimization to be ac¬ 
complished, using any convenient time scale, or alternatively the 
signals which occur during system operation may be used directly. 

Bibliography 
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F. Mesch, Institut fur Regelungstechnik, Darmstadt, Germany 

If the mean square value of the noise n(t) is large compared to that of 
the input signal x(t), the proposed configuration of Figure A might 
fail completely. I want to suggest another configuration that eliminates 
the effect of the noise and, in addition, replaces the extremum-seeking 
(even function of parameter deviation) by a straightforward adjust¬ 
ment to zero (odd function of parameter deviation). 
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Figure A 


Compared with Figure 3 of the paper, the squaring of the difference 
is replaced here by the multiplication of the difference with the input 
signal. It may be shown that to a first-order approximation 

_ __ f 00 o , . dG(jco;K) , 

e(0 = *(0-l>(0-«(0]*AK- S«(a>)- ^ -dco 

where K denotes the parameter to be adjusted, and where the para¬ 
meter deviation AiC is assumed to be small. The integral forms a 
constant and expresses the sensitivity to the parameter K. The first 
factor A£ is proportional in sign and amplitude to the parameter 
deviation, so the model is simply adjusted for A K — 0. This procedure, 

I feel, might be readily extended to more parameters. 

B. P. Th. Veltman, Technological University of Delft, Lorentzweg 1 , 
Delft, Netherlands 

This clear and convincing presentation of the ‘model method seems 
to outperform the ‘statistical method*. The authors’ opinion about 
this point is explicitly mentioned in their paper. If by ‘statistical 
method’ the conventional correlation procedure is meant, this state¬ 
ment needs some more attention. 

The measurement of a correlation function can also be seen as a 
model method: the model being a pure time delay, which is certainly a 
reasonable model if there is complete ignorance about the systems 
dynamics. The output signal of the system and of the time-delay model 
may be highly non-orthogonal, thus necessitating long observation 
times. However, if some dynamic properties of the system are known 
a priori one may use this information for prefiltering the correlator 
input signals to make them more orthogonal. Such a scheme is 
described briefly in the paper by A. van den Bos and myself where 
small bandwidth signals are used as correlator inputs. The parameter 
estimation time is in that case of the same order of magnitude as with 
the model method described by the authors. What I would like to 
stress is that the convergence time in parameter estimation is dominated 
by the a priori knowledge of the system and that it furthermore only 
depends on the available equipment which measuring strategy should 
be followed. 

L. R. Young, Massachusetts Institute of Technology, Cambridge 39, 
Massachusetts., U.S.A. 

I would like to call the authors’ attention to the very efficient method 
for parameter identification at time-varying processes by the multiple 
regression technique, presented by Dr. J. I. Elkind at the Joint 
Automatic Control Conference in New York, June 1962. 

E. Blandhol and J. G. Balchen, in reply 


However, we start with a brief summary of some experimental 
results which were included in the presentation of the paper at the 

Congress: . , . 

Quite recently some preliminary results were obtained from a model 
study of a non-linear industrial process, shown in principle in Figure A. 
A liquid flows through a tube between a pressurized tank and an 
evaporator. The flow is turbulent, giving a quadratic pressure loss due 
to friction. The driving force is the tank pressure minus the vapour 
pressure in the evaporator. The differential equation of this system is 
shown below Figure A. The unknown parameters are m and f; m can 
bestimated fairly well, but the value of £ is rather uncertain. 

The process oscillates, with a period of about 3 min. These oscil¬ 
lations were used as test signals in a model to determine m and £. 



Evaporator 


Figure A. Non-linear industrial process 

Differential equation: 

^ = 1[9.3( Po -p)-0-05UG 2 ] 

d t m 


Time scaling factor: 

P = 115 


Model parameters: 

1 

s 


m 

Expected values: 

0*0264 

122 (uncertain) 

Optimum values obtained: 

0*0274 

200 

Per cent error: 

3*8 

65 

Normalized error measure 

_^min. 





at optimum: 

R = 0*025 



The time scaling factor ft the expected values of the model para¬ 
meters and the values actually obtained are all listed below Figure A . 
It is seen that the optimum value of 1/m is quite close to the expected 
one, whereas £ turned out to be considerably higher than assumed. 
But the normalized error R at optimum is only 2*5 per cent, indicating 
that the model approximation obtained is quite satisfactory. 

Figure B shows the various signals recorded from the system and 
the model, and Figure C shows one of the minimum curves obtained. 
The convergence seems satisfactory in the vicinity of optimum. A 
more general study of the convergence over a broader parameter 
range, or of other model configurations, has not been made yet, but 
these questions will be further investigated. 

Our conclusion to these last experiments is that the model method 
seems to work well also for non-linear systems, probably provided 
certain assumptions regarding signals, structures and convergence are 
satisfied. This is of great practical importance, since most processes 
and systems to be studied will probably contain some non-linearities. 

The above remarks also provide an answer to the question from 
Mr. Hakala. Now we proceed to the different groups of comments. 


We would like to express our most sincere thanks for the great interest 
taken in our paper by all the contributors to the discussion. 

The subjects treated in the discussion may roughly be divided into 
five groups. We have therefore found it convenient to comment on 
each group separately. 


Input and Noise Characteristics 

We certainly agree with Dr. van der Grinten about the sensitivity 
of the model method to input signal characteristics, as also mentioned 
in the paper when discussing the coefficient matrix R for s. But one of 
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the major advantages of the model method, compared with for 
instance, statistical methods or specific transient tests, is, in our 
opinion, that it usually works well with quite short samples within a 
large class of test signals, including the normal process operating 



Our method, and probably any model method, assumes constant 
model parameters throughout each run of the test signal. Thus if 
multiplicative noise due to changes in system parameters is present 
as mentioned by Professor Gorecki, this is equivalent to having a 
time-variable system. The resulting model will then be a mean-square 
approximation to the variable system over the test period. 

System Structure vs. Model Structure 


The main problems here seem to be: 


(1) With model structure equal to a known system structure how 
complicated can that structure be before the convergence deteriorates? 
We believe that not only the order of the system, but also the number 
of adjustable parameters are of importance, since some parameters 
may be known a priori. In general, poorer convergence must be ex¬ 
pected for higher-order systems, containing more parameters but the 
upper practical limit should be 4-5 adjustable parameters. 

(2) What is the effect of using a simplified model for a complicated 
known system structure? It seems impossible to give a general answer 
to this very important problem, although certain conclusions may be 
drawn from theory or from performed experiments. The results 
mentioned by Mr. Paul could thus be readily expected. We find the 
work reported by Mr. Brookes interesting, since any possible reduction 
m the number of model parameters is important. The authors have 
not yet made any experiments on this problem. 


v J™* 1 mod f 1 structure should be used when the system is a 
black box. This is the problem of designing a mathematical model 
as stressed in the paper. One possibility is to try several different model 
structures, and select the one giving the best approximation. In this 
way the model method may be used also for optimizing model struc¬ 
ture, not only the parameters. This would also be the model method 
equivalent to Professor Gorecki’s frequency method for choosing 
between his eqns (1) or (2). 



length 

Figure B. Recorded signals from system and model 




Plotted with 
m = const. 



1 I i | 
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m 

Figure C. Experimentally obtained minimum curve 


(4) What is the difference between a time-domain and a frequency- 
domain model? Here some confusion seems to exist. Theoretically 
the impulse response and the transfer function are essentially equiv¬ 
alent, so that either model might be used. However, in practice we 
are dealing with parameter estimates only, calculated for finite 
measuring time and in the presence of noise. We always want to 
nd optimum estimates, but a method which is optimum in one 
domain is not likely to be optimum in the other. This can be intuitively 
understood by realizing that the frequency-domain model parameters 
are complicated functions of the time-domain parameters, and vice 
versa. Hence it is very difficult to calculate the propagation of para¬ 
meter uncertainties from one domain into the other. Moreover, para¬ 
meter adjustments which may be nearly independent (orthogonal) in 
one domain will certainly not be so in the other domain. We hope this 
provides an adequate answer to Messrs. Niemi and van der Grinten. 

Adjustment Strategy and Convergence 

Also on this point there seem to be some misunderstandings. In its 
present form our model method is not a self-adjusting, on-line method, 
thus the optimization time is not critical. 

. In t J is interestingremarks Professor Feldbaum asks why ourmethod 
is used, since man is a bad optimizer, and a self-adjusting model 
method is not an optimum parameter estimation method. Mr. Barker 
finds no advantage in our method over the use of simple feedback 
loops. If the problem were rapid on-line model optimization, we would 
agree with the two remarks, but when it comes to off-line model 
studies, which have widespread applications, we feel that our method 
has certain real advantages over automatic self-adjusting methods. 
When the total optimization time is not critical, man is in our opinion 
one of the best (and cheapest) optimizers one can have, due to his 
ability to adapt his strategy to the information gained during the 
optimization. As mentioned by Professor Sargent and Mr. Meyerhoff 
one may often encounter response surfaces with local minima or with 
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no convergence towards the desired optimum. In such cases all but 
the most elaborate automatic methods will fail, since they require 
monotone convergence from the starting point towards the optimum. 
But with our manual method, with graphical display of the response 
curves for £, it is easy to get a graphical picture of a broad range of the 
response surface, by plotting a family of e curves for certain values of 
all the model parameters. From a study of these curves is found the 
area within which the absolute optimum probably lies, and a cyclical 
parameter adjustment towards this optimum can then be started. 

Other advantages of our method are: (a) no complex instrumenta¬ 
tion of automatic optimizer; ( b ) the time-scaling provided with ISAC, 
and (c) easy and accurate graphical interpolation of minimum point 
of e curves. 

A requirement on the manual adjustment method is that the basic 
strategy be simple and straightforward, such as the cyclical parameter 
adjustment. Therefore, in reply to Mr. Kumar, the steepest descent 
method is not particularly well suited for manual optimization, and 
has not been tried by us. 

The statement by Mr. Meyerhoff that manual adjustments require 
that the parameters be non-interacting is not correct, although the 
method of course works better if they are. 

Applications to Variable or Non-linear Systems 

In reply to Professor Sargent, we have not yet made any experi¬ 
ments with multi-input models but we believe that if the two model 
channels 1 and II converge independently towards reasonable results, 
the alternate adjustment scheme should converge towards an even 
better result. 

The problem posed by Mr. Rakowski is an interesting but difficult 
one. With unknown structure there is probably no hope for the model 


method, nor in the case of known structure with too many unknown 
parameters. For known structure, and with no more than four or five 
adjustable parameters, it might be possible to optimize these by in¬ 
cluding the complete system with all interconnections and feedback 
loops in the model. A practical limitation is that only three signals 
can be recorded simultaneously on ISAC. 

With regard to non-linear systems, the model method is, in our 
opinion, one of the very few that lend themselves to determination of 
the form of non-linear functions and operators that appear in the 
system. 

Other Methods 

We want to express our gratitude for the valuable contributions 
received from Messrs. Brookes, Kumar, Mesch, Meyerhoff, Reinisch, 
Yeltman and Young. They show that much work is now being done 
in this field, and that many alternative methods exist for solving the 
general problem of process parameter estimation. 

The error criterion proposed by Mr. Mesch works well for one 
parameter, but fails in multi-parameter problems. The first-order 
error approximation then becomes 

_ « f 00 0G 

x(f)'[>(0-«(0]« X A.K t S xx (a>)—d(o 

i= 1 J -oo UiV i 

showing that the adjustment of one AKi to zero does not give zero 
error measure unless all other AKi are zero, which is a trivial case. 
In general zero-seeking error criteria should be avoided, since they 
put certain strict requirements on the magnitudes of additional con¬ 
stant terms in the error expression. Extremum-seeking criteria are to 
be preferred, being independent of constant terms. 
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Les Erreurs Systematiques et Aleatoires dans la Determination 
Experimentale des Fonctions de Transfert 

J. LOEB 

Summary 


Mathematical methods are known by which records obtained in the 
course of normal plant operation yield the time constants of 
transfer functions. Following these methods one systematically tries 
the proposed solutions, with, as a criterion of accuracy, the mini¬ 
mization of a mean square error. 

The paper shows that, if the noise which is added at the output of 
the element under study is correlated with the input signal, the time 
constants which minimize the error are shifted with respect to correct 
ones, hence a systematic error occurs. This precisely happens in 
feedback systems. Moreover, there exists a random error coming 
from noise fluctuations. 

. The P a P er also . sh °ws how to evaluate the systematic errors and 
gives some conditions to be fulfilled by the records so that time con¬ 
stants can be calculated with tolerable errors. 

Cases dealt with are those of transfer functions: 


l + 7> auu l + Tp+T2p2 

Sommaire 

On connait les m6thodes mathematiques au moyen desquelles les 
enregistrements obtenus en fonctionnement normal de l’usine four- 
nissent les constantes de temps des fonctions de transfert. 

Ces methodes comportent l’essai systematique des solutions pro- 
posees, le criterium d’exactitude etant le minimum d’une erreur 
moyenne quadratique. 

k auteur montre que si le bruit qui vient s’ajouter a la sortie de 
1 element etudie est correle avec le signal d’entree, les constantes de 
temps qui donnent l’erreur minimale sont decalees par rapport aux 
vraies, d’ou l’existence d’une erreur systematique. C’est ce qui se 
passe justement dans les systemes a reaction. 

De plus, il existe une erreur aleatoire due aux fluctuations du bruit. 
On indique une fagon d’evaluer l’erreur systematique, et les condi¬ 
tions pour que les enregistrements puissent servir au calcul des cons¬ 
tantes de temps avec des erreurs admissibles. 

Les cas traites sont ceux des fonctions de transfert: 


l + Tp 1 -\-Tp+T2p* 

Zusammenfassung 

Es gibt mathematische Methoden zur Ermittlung der Zeitkonstanten 
der Ubertragungsfunktion anhand von Aufzeichnungen der MeBwerte 
bei ungestOrtem Betriebszustand. Aufgrund dieser Methoden probiert 
man systematisch eine Reihe von vorgeschlagenen Losungen, wobei 
als Mafi fur die Genauigkeit der mittlere quadratische Fehler zum 
Minimum gemacht wird. 

Dieser Beitrag zeigt: Wenn man das Rauschsignal, das am Ausgang 
des untersuchten Elementes hinzuaddiert wird, mit dem Eingangs- 
signal korreliert, dann sind die Zeitkonstanten, welche den Fehler 

minimieren, in bezug auf die wahren Zeitkonstanten verschoben_ 

was das Auftreten eines systematischen Fehlers erklart. Genau das 
passiert in Regelungssystemen. Obendrein tritt ein Zufallsfehler auf, 
der durch Rauschschwankungen bedingt ist. 

Dieser Aufsatz zeigt auch, wie der systematische Fehler ermittelt 


werden kann, und nennt die Bedingungen, die die Aufzeichnungen 
erfulJen miissen, damit sie zur Berechnung der Zeitkonstanten mit 
zulassigen Fehlern dienen konnen. 

Die hier behandelten Falle betreffen folgende Arten von Uber- 
tragungsfunktionen: 

e~Gp i 

1 + Tp Und 1 + Tp + T2p2 


Introduction 

Le present travail se situe dans le cadre des techniques indiquees 
par Milsum 1 . 

On connait la forme des fonctions de transfert, mais on 
ignore les valeurs des differentes constantes de temps et des 
differents coefficients. 

Les grandeurs d entree et de sortie sont enregistrees au cours 
du fonctionnement normal de l’usine; on essaie systematique- 
ment les valeurs numeriques cherchees, en reconstruisant les 
grandeurs de sortie, et on adopte celles qui donnent une erreur 
moyenne quadratique minimale. 

Une telle technique ne donnerait les resultats exacts (avec 
une erreur qui dependrait seulement des appareils de mesure) 
que si le «bruit» etait absent. 

Par bruit nous designons tout l’ensemble de perturbations 
qui agissent sur les grandeurs mesurees, en dehors de celle dont 
on cherche 1’effet. 

Cela pourra etre Taction des autres variables de l’usine, ou 
celle d’elements incontrolables. Par exemple, si on cherche la 
faqon dont la temperature de sortie d’un reacteur varie en fonc- 
tion du debit d’un des constituants a l’entree, 1’effet des varia¬ 
tions des autres constituants sera un bruit. Le refroidissement 
d’une colonne placee a Pair libre, sous 1’effet du vent ou de la 
pluie, en sera un autre. 

Des techniques que nous indiquons, en partant de l’expose de 
Milsum, ne sont utiles que si reellement il y a un bruit important. 

L’Erreur systematique 

Nous allons montrer que dans certains cas (que nous preci- 
serons) l’ensemble des valeurs des constantes de temps T x , T 3 ... 
qui minimise l’erreur moyenne quadratique ne se confond pas 
avec celui des vraies valeurs. Nous donnons ici le detail des 
calculs pour quatre cas simples. 

Cas de Vintegrateur simple 

Soit un element d’usine dont on sait qu’il est representable 
par la fonction de transfert: 

1 

l + Tp 
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On ignore encore T qu’on va chercher a deduire des en- 
registrements suivants (Figure 1 ): 
f(j) grandeur d’entree 
h {t) grandeur de sortie (brute) 

g'(f) grandeur de sortie, non chargee de bruit qui correspond a 
/(/) avec l’hypothese T = T\T' etant la constante de 
temps essayee. 

La sortie accessible h ( t ) est en fait la somme de: g (t) trans¬ 
formee de / ( t ) par 1/(1 +Tp) (inaccessible) et n (t) une perturba¬ 
tion aleatoire. 


ou pratiquement F (t) 




pLif) 

(1 + Tp) 2 


( 7 ) 


Bien entendu, F(t) = 0 si /est une constante. 

Dans cette expression, figure 1’inconnue T mais comme nous 
cherchons un ordre de grandeur, on pourra mettre au lieu de T 


et de T' une valeur du meme ordre de grandeur que nous noterons 
T qu’on doit estimer a priori. 

On a done: 


n = (T' — T) F (t) + i 


(X\ 



Tirons de (8) par elevation au carre et calcul des moyennes: 

^=(r-T) 2 F+2(r-T)F^+^ (9) 

Ici Fn est la valeur de la fonction d’inter correlation 
= F(t — r) n (0 entre F et n pour une valeur nulle du 
decalage r. Lorsque ce terme n’est pas nul, il y a une erreur 
systematique. _____ 

Nous savons deja que r/ 2 va avoir un minimum positif. 

En effet, le discriminant du second membre est: 


L’erreur (dont on cherche a reduire la valeur moyenne 
quadratique) est: 

r\ — h — g r 

Le schema de la Figure 1 s’exprime algebriquement par: 

(a) f=h-n + T(h-n) 

(b) /= g' + T'g' 

Le point sur une expression signifie «derivation par rapport 
au temps ». 

Nous dcrivons maintenant / au lieu d.e/(f) etc— 

Les equations 1 (a) et (b) donnent: 

r,-n + T(h-n)-T'g'=0 (2) 

ou encore en ajoutant el retranchant T h 

r, + T'fi=(.T'-T)h+n + Tn ( 3 ) 

Appliquons maintenant a (3) la transformation de Laplace, en 
notant L (/) la transformee. 

(1 + T'p) L(r,)= (T' - T) pL (ft) + (1 + Tp) L in) (4) 


Maintenant: 



ou: 


.JT-T)pL(f) 
LW_ (1 + Tp)(l + T'p) 


+ L(n) 


( 6 ) 


Appelons F (/) la transformee inverse du premier terme du 
second membre: 




pUf ) 

(X + Tp)(l + Tp) 


(Fri) 2 —F 2 -n^ 

II est toujours negatif (inegalite de Schwarz). 

Mais le minimum de yf quand on fait varier T' n’a plus lieu 
pour r = T. II a lieu pour: _ 

T'= T— 


Or dans un systeme a reaction, le bruit n ( t ) se repercute sur 
la grandeur d’entree f(t). 

Nous sommes ainsi places dans le cas d’application de la 
formule (10). 

Nous devons maintenant voir si cette erreur est importante 
et si un examen plus attentif des enregistrements ne va pas nous 
permettre de la deceler, puis de la corrigen 

Remarquons d’abord que plus le systeme de retour entre 
n(t) et fit) sera complique, plus de sources de bruits non 
correles avec n it) viendront s’ajouter. Done dans les installations 
importantes, il y a suffisamment de sources de bruit pour que la 

valeur moyenne Fn soit petite. 

On a d’ailleurs un moyen de s’en rendre compte par une 


analyse plus serree des enregistrements. 

Nous allons faire porter le travail de depouillement sur deux 

intervalles de temps ou la valeur moyenne quadratique F n est 
pas la mSme. Si en passant d’une portion d’enregistrement a 
l’autre, F 2 se trouve multipliee par un nombre A 2 , l’erreur sur T 

tiree de (2) se trouve divisee par L 

Ce resultat approximatif suppose que n if) est peu correle 
avec F(t), de telle sorte qu’on conservera le meme n (T) dans les 

deux equations (1). , 

On ne trouvera done pas la meme valeur pour T et les deux 

depouillements donneront: 



(ID 


n= 



T' et T\ ainsi que X sont connues. 
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En eliminant Fit) n (t) on trouve 

tJ-FzIL 

1-1 

On trouvera toujours dans l’enregistrement deux periodes de 
temps pendant lesquelles les variations de fit) ne sont pas les 
memes et le calcul (4) sera toujours possible. (Bien entendu si T[ est 
voisin de T{ il n’y a plus de probleme: n {t) n’est pas correle avec 

F{t). De plus 1’ensemble (11) donnera une valeur de F it) n it) 
par elimination de T. 


Fn == F2 fZj( T ^ T ^ 


(13) 


Cas du retard pur (Figure 2) 


f(t) 


- A- Op 

L. /f 

•\ 

fcr r 

9 


1_| 

e - 0, p 

! _ 

7 



9' 


Figure 2 


ou encore 
et 


Si 0 est le retard inconnu, 6' le retard essaye, les relations sont: 

n=h-g’=f(t-d)-f(t-e')+n (14) 

n={9-9’)f(t-9)+n (15) 

F=(9-9') 2 F+F+2(9-9'W (16) 

II n’est plus necessaire ici de preciser que les fonctions sont 
calculees pour l’instant t - 6, car nous les supposons station- 
naires. Comme precedemment la derivee de r/ 2 par rapport 
a 0 - 0' ne s’annule pas pour 0 = 0' mais pour: 


0=h-n-g'-(9-9')g+T(fi-n) 

- T' (g'+(9 - 9') g’) + T'h—T’h 

(12) d’ou: 

n + T Tj = (r - T) h + n + Tn+(9 - 9') [f + T'#'] (21) 

ou encore a cause de (1) 

n+n\T'+(9-9')']+T'{9-9') 

= \T'-T+9-9'~\h+T'(9-9')ii+n+T,i (22) 

Appliquons a (22) la transformation de Laplace. 
[l+(T+9-9')p+T(9-6')p 2 ]l(ri ) 

= [(T'—T+9—9')p+T'(9-9')p 2 ]L(h)+(l + Tp ) L (n) 

p • ^ 23 ) 
Puis comme: 

-MD e~ ep l(r-T+9-9')p+T'(9-9')p 2 1_ 

( 1 + Tp) ( 1 +(r +9 - W) p+r (6- 0 V) w 

(24) 

Les valeurs moyennes ne sont pas changees par la translation 
dans le temps 0 en raison des proprietes stationnaires des fonc¬ 
tions. 

Pour evaluer les coefficients de T' - T z t 0-0' nous 
pourrons negliger 0 — 0' au denominates de (24). Pour exprimer 
L irj) nous aurons besoin, en plus de Fit) definie en (7) de: 


9’=9 --1 

F 

Cas ou il y un retard et une Integration 
La fonction de transfert essayee est: 

„~ 9 P 


(17) 


l + T'p 


(18) 

(19) 


“''Hr+TOo+r,) ou (25) 

Partons de (24) qui, avec les simplifications admises, et en 
negligeant le terme exponentiel qui ne fait que decaler la variable 
t 9 donne: 

t1~FiT'-T+e-6')+PT'i6-0') + n ( 26 ) 

En elevant (26) au carre et en notant que: FF = 0 il vient: 

^=F[r- T+9-9’f+Fr 2 (9 - 9') 2 

+ n 2 +2Fn[T'-T+9-9']+2FnT'(9-9') (27) 
x=T'—T 
y = 6'-0 

En annulant les derivees partielles 


Posons : 


si bien que: 

f(f) = g'(t-9') + T'g'(t-9’) 

avec toujours: 

*l=h-g' 

Faisons dans (19) la substitution: 

g' (t-9') = g' (t - 8) + (9 - 9') g' (t - 9) 
g'(t~6’)=g'(t-9)+(9~9’)g'(t-6) 

trmt^f ranC r° nS (19) de (18) et remar quons que grace a (20) 
outes nos fonctions sont reperees a l’instant (t - 6). De plus 

ajoutons et retranchons T' h 


on a: 


et 

9 y 


drj 

dx 


( 20 ) 


2(x-y)F I +2Fn = Q 

—2(x—y)F r2 —2Fn+2~F 2 T ,2 y — 2 ~FnT '=0 ( 28 ) 

On retrouve bien entendu x = y = 0 si Fn = 0 et si Fn = 0. 
S’il n’en est pas ainsi: 

Fn 


9'=9+ 


T'F 2 


jr_ j, Fn Fn 
~pT' F 2 


(29) 
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Comme dans le cas que nous avons considere precedemment, Toujours avec T'— T — x et 0' — 9 = y, en egalant a zero les 
on levera l’incertitude en prenant deux enregistrements donnant derivees partielles _ _ 


des valeurs differentes pour F° l . 

Cas de la double integration 

La fonction de transfert est du type 

1 


<f(p)= 


1 + Tjp + flV 


et -en supposant 6' — 6 < 6 on obtient les valeurs en T et 0 
qui rendent rf minimale. 

T=T--== 

G 2 


Les temps T, 6 sont les inconnues du probleme. La relation 
entre ft t g est alors (en conservant les memes notations). 


f=g+ T g+e 2 g f 

f=h-n + T(h-n) + e 2 (h-H) 


La fonction de transfert essayee donne: 

f=g'+T'g'+e , 2 g' 


La encore, la technique reposant sur l’analyse de deux inter- 
(30) valles de temps s’appliquera. 

Les erreurs aleatoires 


n -n+T(fi-n)-T'g+e 2 (h-n)-e' 2 g=0 


Ajoutons et retranchons: 


T’h + 9' 2 h 


II vient: 


tj + T'ii + 9' 2 ri = (T'-T)h+(e' 2 -e 2 ) H+n + Tn+9 2 n (32) 
Appliquons la transformation de Laplace 

(1 + T'p+ 0' 2 P 2 ) L 0 7 )= (r - T) P L (h)+(6' 2 - 6 2 ) p 2 L (fc) 


^ Supposons maintenant que la bruit n’est pas correle avec 

l’entree, c’est a dire qu’on a punegliger ou corriger les termes 
-n)-0 g =0 te ] s que pyf ii r estera pour rf des expressions de la forme 

A (T' — T) 2 + B. Le terme B contient n 2 . Si le bruit etait une 
fonction aleatoire rigoureusement stationnaire, les essais four- 
niraient T' sans autre erreur. _ 

En fait, il n’est pas prudent de considerer n 2 comme constant 
K+n + Tn+9 2 n (32) tout au long de l’enregistrement qui sert au depouillement. 

Une valeur essayee de T’ conduit a une fonction g (0 que 
ce l’on compare a la veritable fonction h (f). Si le bruit n’est pas 

, m ,2 a 2 \ 2 r stationnaire, la valeur essayee T (ou surtout 6 s’il s agit d un 

() + (n a )P F(H) retar( j pur ) va nous conduire dans une region de la courbe h it) 

+(1 + Tp + 9 2 p 2 )L(n) qui ne sera pas la meme que celle qui correspondra a la vraie 
valeur T ou 6. 


L ^i + Tp+eV +L(n} 

II vient: 

L(/)-r(T'-T)p+(g' 2 -e 2 )p 2 ] 

W (1 + Tp + Wp 1 ) (1 + Tp +0'V) 

(r-r)p+(9 ,2 - e 2 )jP 2 +i+7>+9y . , 

+ l+rp+fl'y ( 

qui se simplifie en: 

L (/) f(r' - r) p + (g' 2 - 9 2 ) p 2 ] jr () m 


Posons cette fois-ci: 


G (0—1 

ou pratiquement: 


L(f)p 


tTPfp+e^p 2 ) ( 1 + T'pWV) 


r(A L(f)p 
G ^) — (i +ff+e T pf 


rj = (T'-T)G+0 ,2 -d 2 )G+n 

OU 

^=(T'-T) 2 G 1 +(,9' 2 -9 2 ) 2 ~G 1 +n 2 


Evaluation des erreurs aleatoires sur T et T' 

Pour evaluer l’erreur qui serait provoquee par des fluctua¬ 
tions dr? de rc*, nous allons exprimer if en fonction de: 

La fonction d’entree / (t) ou de celles F et G qui s’en dedui- 
sent respectivement par les equations (7) et (25). 

Les fluctuations du bruit. Ces expressions nous sont donnees 
par les calculs precedents. 

Cas d’une seule integration — II suffit de faire dans (9) 
Fn — 0 ce qui donne 

^ = ( T '-T) 2 F + /? ( 38 > 

Cas d’un retard pur 

rj 2 = (9-e') 2 P+r,7 (39) 

Retard et integration 

^ = F^{T'-T+9-9') 2 +Tt' 2 {B-9') 2 +1? (40) 

Double integration 

^2 == (j-^7) 2 G I +(0 ,2 ^0 2 ) 2 ^ + n2 (41) 

Au moment ou on fera les tatonnements sur 1C et 6' il est 
raisonnable de penser que n z aura bouge de d n 2 . On arrive 


_j_ 2 (j" — T) Gn 4- 2 ( 9' 2 — 0 2 ) Gn (36) ainsi aux 4 expressions de Ferreur aleatoire 
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Une integration 


Retard pur 




Retard et integration 


AT + A 6 ~ 



Double integration 


A 6 


1 fSn 2 ¥ 
T 2 \J?J 



Conditions a remplir pour les enregistrements 

(42) L’usage qu’on veut faire de la connaissance des fonctions de 
transfert impose une limite aux erreurs sur Jet 6. La connaissance 
des retards 6 determinera la limite du spectre des perturbations 
entre lesquelles l’usine peut encore etre «contr61able». Celle 

(43) des constantes T determinera la structure (et les valeurs nume- 
riques des composants) des reseaux correcteurs a placer dans 
la boucle de reaction. 

Or il se peut que les amplitudes des variations des grandeurs 
telles <l ue /« ne soient pas assez grandes pour que les equations 
K } ( 42 ) k ( 45 ) conduisent a des valeurs tolerables des erreurs 

A Tet A 6. 

Les fonctions F et G sont, certes, construites au moyen des 
valeurs recherchees Tet 0 mais on en connait l’ordre de grandeur, 
et les equations (42) a (45) peuvent servir de guides pour decider 
si on entreprendra la recherche des valeurs minimales de rf ou 
P our choisir dans les enregistrements les parties les plus 
(4o) favorables. 


A6 ~ 


4 9 : 
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DISCUSSION 


P. Eykhoff, Technological University, Department of Electrical 
Engineering, Kanaalweg 2JB, Delft, Netherlands 

Dr. Loeb’s paper is devoted to the interesting problem of process- 
parameter estimation under the condition that the process-input 
signal and the additive noise are correlated. 

It may be of interest to point out the relation between the opera¬ 
tions used in the paper to derive an expression like eqn (8) of the paper 
and the idea of parameter-influence coefficients, introduced by 
Meissinger 1 ’ 2 . 

These parameter-influence coefficients are the partial derivatives 
of problem variables with respect to system parameters. 



Figure A 

Using the notation indicated in Figure A 

by + y = x ( 1 ) 

fiw + w = x (2) 

e=y-w+n ( 3 ) 

Following Meissinger, the model output w can be considered as 
a function of two variables; w — w (t, /3), therefore 


1fb = p + Ap then y = w(t, p + A/?) and 


y — w = 



(b-PH 




(5) 


For a small difference b — p only the first term at the right-hand 
side of eqn (5) is used. Eqn (5) substituted in (3) then leads to 

fdw\ 

eK \€p)f h -P )+n ( 6 ) 

which is the same as eqn (8) of the paper. 

P arame l ;er influence coefficient can be determined by partial 
differentiation of eqn (2) with respect to f: 



u + u = — w 

(7) 

where 

0W 



ICO 

II 

a 

(8) 


This time-function can be generated as is indicated in Figure A 
It will be clear that it is essentially the same as the function F(t) 
introduced by eqn (7) of the paper. 

This approach has the advantage that the higher-order parameter 
influence coefficients in eqn (5) indicate over what parameter-interval 
the approximation ofeqn (6) holds. 

Minimization of e 2 with respect to ft leads to: 


0=—=2e—=-2e—- 

dp 2e dp 2e dp- 


■ —2ew 


(9) 


With eqns (6) and (8) this leads to the same condition as given by 
eqn (10) of the paper. 
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n 



Figure B 


The paper gives an interesting method of correction for the adverse 
effect of the correlation between process input signal and additive 
noise. One interesting point that this raises in my mind is whether a 
scheme analogous to Figure B could always be used and for what 
reason. 
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The Applicability of the Relay Correlator and the Polarity 
Coincidence Correlator in Automatic Control 

B.P.Th. VELTMAN and A. van den BOS 


Summary 


j 


The increasing need for a statistical evaluation of signals for adaptive 
and optimizing control purposes makes simple, reliable correlating 
devices desirable. In many cases the relay correlator or the polarity 
coincidence correlator can provide satisfactory answers. 

These methods are not restricted to Gaussian signals, but can be 
applied to any type of join t distribution function of the signals by the 
addition of a special auxiliary signal. It is shown that the increase in 
statistical inaccuracy with this procedure can be diminished by taking 
more samples within the given observation time of the signals. A genera¬ 
tor for the auxiliary signal is described. 


The applicability of the polarity coincidence method on Gaussian 
signals may. be restricted by the presence of disturbing signals. It is 
proved that in certain cases the zero crossings and the extreme values of 
the correlation function do not shift with respect to the r axis. Exam- 
pies of the cross-correlation of Gaussian signals disturbed by noise and 
of the detection of sinusoidal signals out of noise are given. Finally the 
capabilities of the polarity coincidence method are illustrated with 
the measurement of the dynamic behaviour of a system in the fre¬ 
quency domain. Two correlating devices whose usefulness has been 
proved are described in an Appendix. 


Die Anwendbarkeit der Polaritatskorrelation auf normal verteilte 
Signale kann durch vorhandene Storsignale beschrankt werden. Es 
wird bewiesen, daB sich in bestimmten Fallen die Nulldurchgange und 
die Extremwerte der Korrelationsfunktion bezuglich der r-Achse 
nicht verschieben. Beispiele zeigen die Kreuzkorrelation von normal 
verteilten Signalen, die durch Rauschsignale gestort sind, und die 
Auffindung von Sinussignalen aus Storgerauschen. 

SchlieBlich werden die Moglichkeilen der Polaritatskorrelation 
erlautert, die sich anhand der Messung des dynamischen Verhaltens 
eines Systems im Frequenzbereich ergeben. 

Der Anhang enthalt die Beschreibung zweier Korrelatoren, die 
sich in der Praxis bewahrten. 


1. Introduction 

A true estimate, for a finite observation time T, of the normalized 
correlation function C (t) between two signals x 1 (t ) and 
x 2 (0 can be obtained by the operation 


Sommaire 

Des correlateurs simples et fiables sont de plus en plus souhaites en 
raison de la necessity d’une evaluation statistique des signaux dans les 
commandes adaptatives et optimalisantes. Dans bien des cas les 
correlateurs a relais ou correlateurs a coincidence de polarite donnent 
des resultats satisfaisants. 

Ces correlateurs peuvent s’appliquer non seulement a des signaux 
gaussiens, mais a n’importe quel type de fonction de distribution 
jomte des signaux par l’addition d’un signal auxiliaire special On 
JJSSLT Im Pf.« C!sion statistique peut Stre reduite en prenant plus 
d echantillons a 1 inteneur de la duree d’observation des signaux On 
decrit un generateur de signal auxiliaire. 

L’efficacite de ces correlateurs dans le cas des signaux gaussiens peut 
etre dunmuee par la presence de signaux perturbateurs. On montre 

fo U net nS rr Cas -, la ValeurP0Ur T = 0et Ies ^eursextremesd'eta 
fonction d autocorrelation ne changent pas de position le long de 

6S 'th S ^ exem P les sont bonnes. Finalement on montre 
que la methode a coincidence de polarite permet de mesurer le com- 
portement frequenciel d’un systeme. Deux correlateurs eprouves sont 


C (t) — < ** (0 * *2 ft ~ f) > T 

<x 1 (t)-x 2 (0>r 

An estimate of the normalized relay correlation function It 

= (0'Sgnx 2 0-T)>r 

<K (01 > t 


(1) 
(f) is 

( 2 ) 


Zusammenfassung 

MCh einer stat *stischen Auswertung von 
S gnalen fur selbsteinstellende und optimale Regelungen erfordert 
emfache und zuverlassige Korrelatoren. In vielen Falten liefert dj 

Re nt^ d t r /m a ^ tatS ’ K01nZ ‘ denZ ' K ° rTeIatorbefriedl g endeI 'dsungen. 

beschrankt sond ^ ^ 1 a" 0111 auf Signale mit Normalverteilung 
oescnrankt, sondern sie finden auch auf Sienalp mit S 

s^fh UnSSfU d n d kti0n Anwendun S’ indem man ein spezielles ffiifs" 
s gnal hmzuaddiert. Es zeigt sich, daB bei diesem Verfahren sesteiaerte 

^ erabgesetzt wird > dae -S 

raiim A t° p ° b ® n des Signals in dem gegebenen Beobachtungszeit 
raum entmmmt. Em Generator fiir das Hilfssignal wird beschrfeben. 


An estimate of the polarity coincidence correlation function* 
P (t) is 

P\V=< sgn Xj (f) • sgn x 2 {t—x)> T ( 3 ) 

‘Sgn’means that the polarity compared to the average value of 
the signal has to be taken, t is a timeshift and < > T is the 
average value over the observation time T. 

n\ bVi0U \ that the instrument ation for simulating eqns 

tinned 3 - mUCh Smpler than for eqn W- Timeshift, multiplica¬ 
tion and integration are easily instrumentated with 1 bit signals 

t 7 ons P inT T mtheAppendix - Several less rigorous simpfifica- 
h! L ° perat,on are P°f ible w iA acceptable extensions as to 
Sth thTm C ° mp eX i ty ; h0wever ’ the important advantages 

t tr rm6 t nd of 1 w t are l 

, b , h other methods. Although several authors in the past 

the n coT Significant simplifications, the n.r.c.f. and 

ering f ’ method s are not m common use in control engine- 

fnrK??? Vely c 0n p argUeS that the simplifications must be paid 
for by a loss of information. The relative short observation times 
that are usual in control engineering do not permit any waste 

n c*f N ™f d d COrrelati ° n fUDCtion wm now be abbreviated by 
’ norm alized relay correlation function by n.r.c.f.- polaritv 
coincidence correlation function by p.c.c.f. ’ ^ 
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of information. It can be shown, however, that this intuitive 
reasoning does not hold in a number of cases. 

The following relations have been proved 1-4 . 


(/) Calculation of the normalized correlation function from 
the normalized relay correlation function or from the polarity 
correlation function for stationary Gaussian ergodic signals with 


zero means: 
n.r.c.f. 

p.c.c.f. 


C (t) = R (t) 


C (t) = sin 



lim r r (t) = R (t) 

T -*■ oo 


(4) 

(5) 

( 6 ) 


lim p t (t) = P(t) (7) 


(//) The variances in the outcome of eqns (2) and (3) are of 
the same order of magnitude as those of eqn (1) for equal 
observation time T. In all cases the variance depends upon the 
fourth product moment of the signals (see also McFadden 5 and 
Fuller 0 ). 


(///) The n.r.c.f. and p.c.c.f. methods are applicable to any 
ergodic process if a special auxiliary signal is added before the 


polarity i: 

s taken. 


n.r.c.f. 

C(t) = T°-.R(t) 

<Tq 

(8) 

p.c.c.f. 

C(t)= M. P ( I ) 

0b<7T 

(9) 


ci l are the variances of the signals that have to be correlated. 
The auxiliary signal must have a uniform amplitude density 
function between the extreme values A of the signal. The 
correlation function of the auxiliary signal must be an approx¬ 
imation of the S function. 


(iv) With eqns (8) and (9) the variances in the estimation of 
C (r) can be significantly larger than an estimation according 
to eqn (1). 

From statement (if) it can be concluded (and this has been 
extensively verified by experiments) that for Gaussian signals 
the n.r.c.f. and p.c.c.f. methods are not merely ‘poor man’s 
solutions’, but they have almost the same power as the n.c.f. 

method. ., _ 

According to (/) and (if) it seems to be of no use with Gaussian 

signals, to apply more complicated devices than the polarity 
coincidence correlator. Restrictions must be made, however, 
when disturbing signals are present. Suppose that the correlation 
between signals x 0 (/) + x„ (0 and * (r) is measured (.Figure!)^ 
the correlation between x t <j) and x 0 (t) should be establishe . 


x i J 

System 

1 * f x n + x n 

1 *0 JL + °*. n 


1 



Correlator 



Figure L 


Measurement of the cross-correlation function between x t (t) 

and x 0 (t) if a disturbing signal x n (0 is present 


The presence of signal x n (t) has no implements for the n.c.f. 
and the n.r.c.f. method. 

According to eqn (4) < sqn (t — r) •[ x Q (t) + x n (t)] > T 
is proportional to < x t (t — r) • [x 0 (t) + x n (/)] > t which may 
be replaced by 

< X; (t — t) • Xq (t) > r + < X; (t — t) • X„ (!) > T 

This additive rule does not hold for the polarity coincidence 
method, due to the sine relation (5). It is shown in Section 3 of 
this paper under what conditions the p.c.c.f. method is still 
useful with added disturbances. According to (iv) above, the 
application of the n.r.c.f. and the p.c.c.f. seems to meet 
restrictions for non-Gaussian signals, if only a short observation 
time is permitted. In Section 2 it is shown that restriction (iv) can 
be weakened by increasing the number of samples within the 
given observation time. A generator for the auxiliary signal, 
that will permit such an increase, is described. 

In Section 4 the capabilities of the p.c.c.f. are illustrated 
with an unusual method for the on-line measurement of some 
dynamic parameters of a system in the frequency domain. 


2 (a). The Variance when the Auxiliary Signal is Added 


The addition of the auxiliary signals makes the n.r.c.f. and 
the p.c.c.f. method applicable to any type of joint distribution 
function. It should be noted that it is not necessary to add 
different auxiliary signals; if the same auxiliary signal is added to 
all signals (e.g. with multiple correlations) it only makes the 
values of the correlation functions for x = 0 undetermined. 
According to Veltman and Kwakemaak 2 , the variance in the 
p. c. c. f. when auxiliary signals are added is, for a sufficiently large 
time shift, approximated by 


var ^ T)= ipT‘ 


, , , 1 * 0*1 
•varcW+^ll-^ 


( 10 ) 


ff 2 are the variances of the signals; A’s are the extreme values of 

the signals. . . 

Var c (r) is inversely proportional to the observation time 7. 
N is the number of samples within the time interval T. It would 
be preferable to use in (10) the bandwidth of the auxiliary signal 
instead of the number of independent samples N within time T. 
A description of the constraints to the power spectrum of the 
auxiliary signal equivalent to ‘succeeding samples must be 
independent’ is, however, rather involved. 

The right-hand side of eqn (10) can be divided into a con¬ 
tribution from the statistical properties of the signals themselves, 
and one introduced by the auxiliary signals. The comparison of 
eqn (9) with eqn (10) shows that, although the measured value 
p (t) is smaller than the wanted value c 00, the relative inaccuracy 
due to the statistical properties of the signal alone is unaffected. 

An increase in the number of samples N will decrease the 
statistical errors due to the addition of the auxiliary signals 
(provided that succeeding samples of the auxiliary signals remain 
independent). The question is, for what value of N is the influence 
of the auxiliary signal acceptable? ‘Acceptable’ can be expressed 
in terms of relative contribution to the total variance of the first 
and second terms in the right-hand side of eqn (10). 

The uncertainty of an estimate can be described by 

[var c(r)]i I c(r). 
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Suppose [varpQ)]* [~var c(t)1* 

P(0 ~ P ' c(x) 

with /? as a measure for the increase in uncertainty with the 
addition of the auxiliary signal. 

The substitution of eqns (9) and (10) in this expression, and 
putting a 0 /A 0 = a/A 1 = a results in: 


/? = 



a 4 iVvar c(t) 


li 


a 4 (/? 2 —l)var c(t) 

Realistic values of the parameters are a = 1/3; var c (r) = 0*001 
for N ' = 10 4 (this strongly depends on the form of the correlation 
function). This means that the addition of the auxiliary signal 
leads to = 3. To reach the value /3 = 1-5, more than six times 
as many samples are needed. If var C(r) = 10~ 3 for N = 10 6 , 
the increase in inaccuracy with the addition of the auxiliary 
signal is only 35 per cent. 


2 (b). A Generator for the Auxiliary Signal 

To keep succeeding samples of the auxiliary signals independ¬ 
ent a high frequency generator for a random signal with uniform 
probability density function is needed. It is proved by Wonham 
and Fuller 7 that signals with a variety in probability density 
1 unctions can be generated from a binary signal, in which the 
number of transients in a certain time interval obeys a Poisson 
distribution, by passing this binary signal through a first-order 
low pass filter. 

It is usual to derive the Poisson binary signal from a radio¬ 
active source. The occurrence of a count from a Geiger-Muller 
tube or a scintillation crystal triggers a bistable circuit. Another 
approach, which leads to simpler equipment, is the derivation 
of the binary signal from Gaussian noise. The problem is how 
to shape the frequency spectrum of a gaussian signal in such 
a way that the distribution of the number of zero crossings of the 
Gaussian signal in a certain time interval can be approached by 
the Poisson formula. It may be assumed 5 that this is approximate¬ 
ly achieved when the second moment of the probability density 
function of the clipped Gaussian noise equals the second moment 
of the Poisson on-off signal. This implies that the p. c. c. f. of the 
shaped Gaussian signal should be identical to the n. c. f. of the 
Poisson signal. 

The n.c.f. of the Poisson signal is 9 


C(r) = e 22,t| (12) 

m which A is the Poisson parameter. The n.c.f. of the shaped 
Gaussian signal must be, according to eqn (5), 


C(r) 


— sin - 


2 e 


(13) 


If (13) is developed into a McLaurin series and Fourier trans¬ 
formed to the frequency domain, it follows that the power 
spectrum of the shaped Gaussian signal is 


|X(m)| 2 = cr 2 -£ 

n — 0 


2 *(— 1)” 


2n+ 1 


22 


(2 n)\ 

which is a non-negative function. 


(2 n +1) 2 (2 A) 2 + co 2 
(14) 


Eqn (14) can be approximated by the first three terms of the 
series. After some modifications in the expression it follows 

I y 12 _ ^2 .2 7cl (3145 2 4 +18 *2 X 2 co 2 + 0*024 co 4 ) 

|Z(C0)I (41^+co 2 )(36A 2 +m 2 ) (1 00 J 2 + C0 2 ) < 15 > 

which is realizable with operational amplifiers and RC circuits 
acting on wide band noise. The procedure gives a signal which, 
after clipping, has an auto-correlation function of the form 
e~ 2 *W 

Experimentally it is established that simpler filters can be 
applied to the Gaussian signal, which also result in a signal with 
uniform amplitude density function after clipping and first-order 
low pass filtering. Satisfactory results are obtained by shaping 
wide-band noise with a first-order lag filter. The time constant 
of the second first-order filter must be about three times as large 
as the tin's constant of the first one to obtain a linear cumulative 
probability function. Figure 2 shows some results. It should be 
noted that succeeding intervals between zero crossings of 



(b) ( C ) 


Figure 2. Cumulative probability functions of signals that are obtained 
by first-order lag filtering (time constant 0 2 ) of a clipped Gaussian 
signal. The Gaussian signal is obtained by first-order lag filtering (time 
constant $ 1 ) of wide band Gaussian noise. Samples of the corresponding 
signals for dfd, = 0(a). 6J6, = 3(5) and 0J6, = 30(c) are given 


Gaussian signals are not independent. No true Poisson on-off 
signal can be obtained by filtering and clipping a Gaussian signal; 
although first and second moments of the signals are made the 
same, the higher moments must differ 5 . 

3. The Polarity Coincidence Correlation Method Applied to 
Disturbed Signals 

(a) Cross-correlation between Gaussian Signals under the Presence 
of Added Noise 

To be able to interpret the result of the polarity coincidence 
method in this case it would be necessary to know the joint 
probability density function between the input x t (f) and the 
output x 0 (t) + x n (?) in Figure 1. This is not possible in general. 
A sophisticated method can be applied, however, for determining 
the average probability that both input and disturbed output 
have the same polarity 8 . 
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pp 


The joint probability density function between the undisturbed 
signals is integrated over an interval with the disturbing signal 
as a time-varying boundary. Averaging out in the time after 
integration results in an expression for the average probability 
for equal polarity of the disturbed signals. For Gaussian signals 
Xi (t) and x 0 (/), with zero mean and variances <r 2 , the procedure 
is as follows. The joint probability of the undisturbed signals is 


1 

-— . exp 

27J<7,<7o(1-C 2 )* 


x 2 2 c x t x 0 + x 0 2 
_ Z (1-C 2 ff 0 J 


(16) 


in which C is the normalized correlation coefficient. If (16) is 
integrated over the interval [— n (t), oo; 0, oo] the probability 
for coinciding positive polarities is calculated from 




2nc i c 0 {l-C 2 f. 


-n(t) 


Xi 2 — 2 CxiXo + xo 2 

e 2 ff,<r 0 (i-c 2 ) • dx 0 * d X t 


(17) 


P(t> 


- arc si 


. , . COS 007 —C (t) c 

sin C (t) H --- r-r ^ 


(1 — C 2 (t)) 


72 


coscor-C^) [c oso>t + C 2 (r)-2]^-+... 

(1 — C 2 (t)) /2 


( 20 ) 


with S — <z 2 /2 <r 2 . 

If C (r) = 0 (for sufficiently large r) eqn (20) reduces to 


P(t)=— cos tor 

71 


\ s Nl 

_ 2 


/S 3 

+^-( cos2 cot + 6) 
36 


48 


(cos 2 COT + 2) + .. 


The n.c.f. for large values of r is 
C(T) = 


1 + s 


•COSCOT 


( 21 ) 


( 22 ) 


By series development of this integral with respect to n (t) 
[assuming that the odd moments of the probability density 
function of n (J) are zero] and averaging out in time, it is found 
that the p.c.c.f. (which is equal to < 4 P ++ — 1 >t) is given by 



arc sin C (t) 


C(T) 

2!(1-C 2 (t))‘ /2 


<n 2 (t)> 

-_ 


Comparison of eqn (21) with eqn (22) shows that the zeio 
crossings of the n.c.f. and the p.c.c.f. are identical. Taking the 
first derivative in eqn (21), and substituting co r = (2 Ic + l) n/2 


gives 


P 1 (T) = (-1)' 


k+1 


2 CO 


0 S 2 S 3 

S — 2 + ~6 


^4.^1 

24 120 


C (t) (3 — 2 C (t)) 2 < n 4 (t) > i (lg) 

4!(1-C 2 (t)) 3/2 " J 

This is identical to (5) if n (t) == 0; (5) forms a good approxima¬ 
tion for (18) if [< n 2 (/) > ] a~ 2 < 0* 1. It is evident that P (r) will 
be zero if C (t) is zero. Thus the zero crossings of the n.c.f. and 
the p.c.c.f. are identical. Differentiation of (18) with respect 
to t and substituting dC/dr = 0 shows that the abcissae of the 
extreme values of the n.c.f. and the p.c.c.f are the same. 
Figure 3 illustrates these statements with some measurements. 

The applicability of the polarity coincidence method seems 
rather limited with disturbed signals. Interesting information 
about a system’s dynamic behaviour can, however, be obtained 
from the shifts in zero crossings and extrema between the input 
autocorrelation and the input-output cross-correlation function, 
as is shown in Section 4. 

(b) Restoring Sinusoidal Signals out of Added Gaussian Noise 

This problem can be tackled in the same way as the procedure 
described in Section 3 (a). In this case P ++ is given by a time 
varying integral of the bivariate probability density function of 
the gaussian noise (zero mean and variance a n 2 ): 


Eqn (19) 

Series development with respect to a sin o)t and averaging out 
in time results in 



Figure 3. Measured cross-correlation functions sinO/2*/?(r)] between 
input and output of a third-order system. The position of the zero 
crossings and the extrema is unaffected when noise is added . The 
third-order system consists of three equal time constants of 0T sec. The 
input signal is a Gaussian random signal with a flat spectrum between 
0*2 and 2 c/sec (dotted line is the input correlation function ) 

q— o—O without noise 

£_• signal-to-noise power ratio 1*55; the noise has a flat spec¬ 

trum between 0*2 and 2 c/sec 

Sampling distance is 44 msec; observation time 10 5 samples — 4,400sec 


*Eqn (19): 


p ++ =- 


2na„{l — C) 


2\‘/2 


— a sin at J 


X n — 2CXnXn-c ‘^ x n x 

2<7 n 2 (i-c 2 ) * dx„dx„ 


-a sin co (t — r) 


(19) 
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which is equal to 

dPfr) 

d t 


=( _l ) ft+1 ^ 0 [ 1 _ e _ S ] 


The abcissae of the extreme values are thus identical with the 
n.c.f. and the p.c.c.f. if r is sufficiently large. It should be noted 
that the signal to noise power ratio S can be estimated from the 
intersection point of the tangents in succeeding zero crossings 
of the p.c.c.f., according to eqn (23). Figure 4 shows the experi¬ 
mental verification of these derivations. 


4. Measurements of the Dynamic Properties of a System with the 

Polarity Coincidence Method 

Measurements for evaluating dynamic parameters of 
systems are presented as an illustration of the applicability of 
the p.c.c.f. in automatic control. It should be noted that any 
estimate of a correlation function, however bad it may be, always 
shows a very true appearance. This deceitful result is due to the 
fact that a correlation function is not a set of independent 
correlation coefficients. There exists a strong dependency between 
the items of a time series representing a continuous signal. This 
dependency causes a correlation between the variances in 
neighbouring points of the correlation function; apparently 
periodic components, that are not present in the signal, may be 
introduced by this phenomenon. 

Thus it is necessary to calculate confidence limits with each 
measured correlation function to give an adequate interpretation. 
The measurements presented in this article are always extended 



over a very long period; in most cases the confidence limits 
(95 per cent certainty that the true value is within the given 
interval) approach line thickness in the figures. Figure 5 gives 
the measured cross-correlation function of the input and output 



Figure 5. Measured cross-correlation function sin (n/2) • p (r) between 
input and output signal of a first-order low pass filter for different band- 
widths of the input signal. Time constant of the filter 0*3 sec; sampling 
distance 44 msec; observation time 10 5 samples. 

Bandwidth of the input signal 

O—O—O 0-5 c/sec ( auto-correlation function of this signal is 
+—f*—+ 0-25 c/sec presented as a dotted line) 

#—#■—# 0-100 c/sec 
No extra noise is added 

signal of a first-order low pass filter, the input signal being a 
Gaussian random signal with a flat power spectrum. No extra 
noise is added. Deconvolution of the cross-correlation function 
with the auto-correlation function produces the pulse response 
of the system. The tedious deconvolution can be simplified by 
using wide-band noise for testing the system. The measurements 
of Figure 5 show indeed that the cross-correlation function 
approaches the system’s pulse response if the bandwidth of the 
input signal is increased. It should be noted, however, that the 
amplitude of the measured normalized cross-correlation function 
deteriorates with an increasing bandwidth of the input signal. 
Only a small part of the input power spectrum appears at the 
output, making the value of the normalized cross-correlation 
small. This inaccuracy is not due to statistical inference but to 
the fact that two large numbers, the integrals of the positive and 
the negative result of the multiplication, are subtracted, giving 
a large measuring error in the difference. 

More accurate results are obtained if the signals that have 
o e correlated cover the same part in the frequency domain. 
Suppose, for example, that input and output of the system are 
faltered by two identical small bandwidth filters (Figure 6). The 
output of the filters is shown in Figure 7. The wave-form of the 
fUtered output signal seems to be shifted in time with respect to 
the filtered input signal. This is due to the almost linear phase 

shift and constant gam of the system for a small bandwidth 
signal. 

The correlation functions describing the small bandwidth 
behaviour of a thud-order system, measured by the polarity 
method, are given in Figure 8. Indeed the phase-shift of the 
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Noise 



Figure 6. Measurement of the dynamic properties of a system by small 
bandwidth filtering and correlating 


wyyw, oflifrtf 

(c) 

Figure 7. Output signals of the filters in Figure 6. Filter decay is 
24 dB/octave to both sides of the centre frequency, {a) Upper trace is 
filtered input, lower trace is filtered output; (b) clipping after filtering. 
Dotted curve is filtered input, drawn curve is filtered output, (c) as ( ), 
the filter centre frequency approaches the — 180° phase shift frequency 
of the system 



third-order system for the particular frequency can be measured 
very accurately, even under the presence of disturbing noise. 
It is possible in this way to make on-line measurements of the 
phase characteristic of a process. 

A follow-up system for tracking the frequency with - 180° 
phase shift in a system has been investigated; the procedure 
shows the rapid responses to changes in the dynamic properties, 
that are characteristic for model methods. This follow-up system 
resembles the well-known lock-in detectors that are used in 
radar techniques for estimating the time delay between an input 
wave-form and a distorted output wave-form. 

Appendix I—A Parallel Correlator (Relay Type) 

Based on the block diagram in Figure 9 a correlator has been 
built from commercially available logic modules. The sign of a 


signal x 1 is transported in a shift register delay line. Electronic 
relays can be connected to every shift register stage. These 
switches pass other signals, provided with the right polarity 



Figure 9. Block diagram of a parallel correlator 
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depending on the state of the corresponding shift register stage, 
to integrators. An auxiliary signal can be added at the entrance 
of the clipping circuit. 

The device consists of 50 shift register stages, 25 electronic 
switches, 3 inverters, 3 clippers with entrances for auxiliary 
signals and 10 binary dividers for producing sub-frequencies 
of the shift-pulse frequency (maximum frequency with the 
used components is 200 kc, allowing a minimum time distance of 
5 psec between succeeding points of the correlation function). 
These units can be interconnected with a patch panel to pro¬ 
gramme the device for calculation of auto-correlation function, 
multiple cross-correlation with positive and/or negative time 
shift, etc. It is evident that this set-up can be simplified to a 
polarity coincidence correlator by using AND gates with 
counters instead of electronic relays and integrators. 

Appendix II—-A Serial Correlator (Polarity Coincidence Type) 

This device has been developed for measuring multiple cor¬ 
relations in slow processes. The polarities of the signal samples 
are recorded in paper tape according to the diagram in Figure 10. 


AuxiLsignal Trip pulse 


1 

Clipper 

1 , 

i 



I_ 

8 track 
puncher 

- Tape I 

- Tape 11 


Figure 10. Block diagram of a device for simultaneous recording of the 
polarity of eight signals with a paper tape puncher with double tape 
attachment 

A positive polarity appears as a hole in a certain track. Eight 
parallel tracks are possible. Sampling speed ranges from one 
every 10 sec to 33 samples/sec. Two identical tapes are punched 
simultaneously. A tape may contain 100,000 samples of a signal 
in a single track (300 m paper tape). 

The reading device consists of a bi-directional punched tape 
transport mechanism that transports both tapes parallel and 


keeps them synchronous with free running, mechanically coupled 
sprocket wheels. A photo electric reading head gives pulses to a 
coincidence detector, coincidence of ‘no holes’ being determined 
with the sprocket hole. Very high speeds are possible (over 
4,000 symbols/sec) as neither constant speed nor immediate brak¬ 
ing is necessary. Delay is obtained by shifting the sprocket 
wheels electromechanically with a ratch wheel construction. 
This can be done while tape is reversing direction. Counters with 
an automatic print out display the results. A typical example of 
the scope of the instrument is: 100,000 samples of four signals 
(three inputs and one output) are collected in 24 h. Reading off 
four auto-correlation functions and three cross-correlation 
functions each consisting of 50 points, is possible within 1 h. 

Several hundred correlation functions have been determined 
with these devices. By and then verification with a digital com¬ 
puter on 2-decimal data have been carried out, with very satis¬ 
factory results in all measurements. 
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DISCUSSION 


P. Jespers, University of Louvain, Belgium 

In the case of multiple input signals, the addition of independent 
auxiliary functions in the same number as there are input signals 
provides a correct correlation function for any value of r, whatever 
the joint probabilities of the input signals may be. Such function 
generators can be easily realized in analogue form by means of 
synchronized sawtooth generators, and in digital form by means of 
quasi-random number generators. 

B. P. Th. Veltman, and A. van den Bos, in reply 

We thank Professor Jespers for his clarifying remark concerning the 
addition of the auxiliary signal. To eliminate the influence of the 
auto-correlation of the auxiliary signals for r = 0, it is indeed 
advantageous to add different auxiliary signals to the ones that have 
to be correlated. We disagree, however, that these auxiliary signals 
are orthogonal. 


In fact, the use of deterministic orthogonal auxiliary signals will 
decrease the additional statistical variance. However, the deter¬ 
ministic orthogonal signals will always have a periodic character in 
order to be able to generate them easily; this necessitates some care 
with regard to the sampling period. 

In an example of an auto-correlation function of a Gaussian signal 
with and without auxiliary signals, the variance hardly increases with 
the additional signal; proper normalization for t = 0 will also 
decrease the variance. The auxiliary signals in this case are simply 
two triangular waveforms of period t x and 2t t . 

J. Delcour, Applied Scientific Research Organization, Lorentzweg, 
Delft , Netherlands 

In your paper it is stated that the statistical variance of the polarity 
coincidence procedure is of the same order of magnitude as if the 
full amplitude range of the signal was taken into account. Can you 
show any experimental verification of this ? 


626 




THE APPLICABILITY OF THE RELAY CORRELATOR AND THE POLARITY COINCIDENCE CORRELATOR IN AUTOMATIC CONTROL 


B. P. Th. Veltman, and A. van den Bos, in reply 

In Figure 2 we show an example of the variances in an auto-correlation 
function for increasing quantization roughness. These measurements 
were obtained as follows: with a commercial digital voltmeter, a 
recording with seven bit accuracy was made on punched tape in a 
pure binary code from a Gaussian signal. Two identical tapes were 
punched simultaneously. 

With the tape reader described in Appendix II the correlation func¬ 
tion was measured with the aid of a special digital multiplier. This mul¬ 
tiplier works as follows: a number from tape I is read in a counter; 
a gate is opened and the counter is emptied by pulses from a 1 Me 
generator, then the gate is closed. This procedure is repeated as many 
times as the number from tape II indicates. The total number of 
pulses of the 1 Me generator passing through the presettable counter 
is measured. By leaving out the less significant bits one is able to 
calculate the correlation function for increasing quantization 
lroughness. 

This way of multiplication and integration proved to be much 
simpler than binary multiplication and could be realized with relatively 
few (20) commercial logic circuits. 


H. Kwakernaak, University of California , California, U.S.A. 

It is not necessary to make the derivation of section 3(a) as compli¬ 
cated as it is. If x 0 (t) and x n (t) are both Gaussian, then x 0 ( t ) + x n (0 
is also Gaussian. 

Therefore the p.c.c.f. of x% (t) and x 0 (/) + x n (0 is given by 

2 

^Xi, Xq + X n CO ^Xl, XQ + X n (f) (^) 

where R Xh Xo + Xn (r) is the normalized cross-correlation function. 
If Xi (t) and x n (t) are uncorrelated, this normalized cross-correlation 
function can easily be found to be 


R 


xi, xq 4 " X-) 


;„(*) = 


K xi , xo (r) 


fK XUXi (0)[K XOtXo (0) + K Xn , Xn ( 0)] 


( 2 ) 


where K XiiX0 (r) represents the (not normalized) cross-correlation 
function of xi{t) and x 0 (t), etc. It is simple to find that (a) can be put 
in the form , R (t) 

P Xi ,x o+*„0)=— arc sin --- 


/ t —j— ^ Xn> Xn 


( 0 ) 


K, 


Clearly, by determining 


( 0 ) 


sin ~-P XiiXa+Xn (T) 

one finds a function which is proportional to the desired cross¬ 
correlation function R Xi> Xo (t). 

The proportionality factor depends upon the signal-to-noise ratio 

^o.xo(O) 

K Xn , Xn ( 0 ) 

It can be seen quite clearly from Figure 3 that this phenomenon 
indeed occurs. 


B. P. Th. Veltman, and A. van den Bos, in reply 

Dr. Kwakernaak’s remark is much appreciated. In the case of Gaussian 
disturbances a direct proof of the applicability of the polarity method 
with cross-correlation can indeed be given. Although we were aware 
of this, we never verified the result in that case. It is very interesting 
to see that the correlation function will only be multiplied by a constant 
factor. 

This gives certain possibilities for estimating the power of the 
noise compared to the signal power. A situation occurs accordingly 
with the addition of auxiliary signals, where the known power of the 
auxiliary signal admits an estimation of the signal power. 
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Summary 

The study of systems increasingly requires the use of random functions 
to represent both imputs and noise. In the automatic control field, the 
use of statistical criteria makes possible the computation of the eco¬ 
nomic efficiency of the system when working under conditions close 
to real operation. 

This paper attempts to define a process generating a random func¬ 
tion having some properties which give it a ‘physical aspect’: limited 
maximum acceleration, limited amplitudes, or still better, probability 
density of given amplitudes. 

Preliminary tests lead to definite functions with constantly maximum 
acceleration (positive or negative): the construction of such functions 
does not present difficulties and one can use three parameters to fit 
the curve to the physical phenomena to be simulated; however, it is 
not possible to impose a probability density in amplitude. 

Finally, a generating process is given for the imposed maximum 
acceleration, the probability density (curve) and at least a parameter 
which is used to adjust the time scale. The function is made of a 
succession of parabolic arcs with a common tangent at their points of 
junction. Samples have been built as well as the corresponding auto¬ 
correlation function. The direct computation of some mathematical 
functions (particularly the autocorrelation function) attached to these 
random functions is being studies. 

Sommaire 

L etude des systemes implique de plus en plus V usage de fonctions 
aleatoires pour reprSsenter soit les entrees, soit les bruits. En Auto- 
matique Industrielle, l’usage de criteres statistiques permet de calculer 
le rendement economique du systeme dans des conditions proches du 
fonctionnement reel. 

, Cette note a P° ur but de definir un processus generateur de fonction 
aleatoire ayant certaines proprietes qui lui conferent un «aspect physi¬ 
que)): acceleration maximale bomee, amplitudes bornees, ou mieux, 
densite de probabilite en amplitude imposee. 

Des essais preliminaires ont conduit a definir des fonctions a ac¬ 
celeration toujours maximale (en module): la construction est simple 
on dispose de 3 parametres pour «ajuster)) la courbe mais on ne peut 
pas imposer a priori une densite de probabilite en amplitude. 

On donne alors un processus generateur pour une acceleration 
maximale imposee, une densite de probabilite imposee (courbe) et au 
moms un parametre pour ajuster l’echelle des temps. La fonction est 
constitute par une succession d’arcs de parabole se raccordant tan- 
gentiellement. Des echantillons ont ete construits ainsi que la fonction 
d autocorrelation correspondante. Le calcul direct de certaines fonc¬ 
tions mathematiques, la fonction d’autocorr61ation en particulier, at¬ 
taches a ces fonctions a!6atoires est en cours d’etude. 

Zusammenfassung 

Die Systemuntersuchungen erfordern mehr und mehr den Gebrauch 
von Zufallsfunktionen zur Darstellung sowohl der Eingangs- als auch 
der Rauschsignale. In der industriellen Automatisierung gestattet die 
Verwendung statistischer Kriterien die Berechnung der Wirtschaftlich- 
keit ernes Systems unter wirklichkeitsnahen Bedingungen. 

_ Arbeit mac ht den Versuch, ein Verfahren zur Erzeugung einer 
Zufallsfunktion zu definieren, die den physikalischen Bedingungen 
moglichst nahekommt: eine beschrankte maximale Beschleunigung 
begrenzte Amplituden oder besser eine Wahrscheinlichkeitsverteilung 
der gegebenen Amplituden. 


Vorlaufige Untersuchungen ffihrten zu Funktionen mit stets maxi- 
maler positiver oder negativer Beschleunigung. Die Erzeugung einer 
solchen Funktion ist einfach, man verfiigt fiber 3 Parameter, urn die 
Kurve der simulierten physikalischen Erscheinung anzupassen, aber 
es war nicht moglich, von vornherein eine Wahrscheinlichkeitsvertei¬ 
lung der Amplitude vorzugeben. 

Es wird daher ein Verfahren zur Erzeugung von Funktionen bei 
vorgegebener GroBe der maximalen Beschleunigung und der Wahr¬ 
scheinlichkeitsverteilung der Amplitude sowie eines Parameters zur 
Anpassung des ZeitmaBstabes, vorgeschlagen. Die Funktion wird 
durch eine Folge von Parabelbogen, die sich tangential aneinander 
anschlieBen, gebildet. Beispiele wurden aufgestellt, sowie auch die ent- 
sprechende Autokorrelationsfunktion. Die direkte Berechnung ge- 
wisser mathematischer Ausdrficke auf Grund dieser Zufallsfunktionen 
wird noch untersucht. 


Objet 

L’objet de cette note est de definir un processus generateur d’une 
fonction aleatoire satisfaisant les conditions suivantes: (a) densite 
de probabilite des amplitudes donnees; (b) acceleration (derivee 
seconde) maximale donnee en module; c’est ce caractere qui 
confere principalement l’aspect physique de la fonction cherchee; 
(c) possibility d’ajustement de l’echelle des temps (il s’agit 
d adapter au mieux le spectre de la fonction cherchee); (d) prin¬ 
cipals proprietes statistiques calculates (fonction d’auto- 
correlation, spectre, etc....), et ( e ) echantillons facilement 
reproductibles. 

Interet de ces fonctions 

Habituellement, 1’etude des systemes se fait a partir de 
sollicitations sinusoidales. Si le systeme etudie est lineaire il 
suffit d’etudier le rapport des amplitudes de l’entree et de la 
sortie, et le dephasage entre Tentree et la sortie, pour toutes les 
frequences du domaine a etudier. Mais si le systeme n’est pas 
lineaire, pour chaque frequence, il faut etudier la reponse a 
diverses amplitudes. Bref, il faut explorer un domaine du plan 
frequences/amplitudes et de ce fait l’etude du systeme est 
beaucoup plus complexe. L’etude experimentale des reponses a 
des sollicitations sinusoidales necessite la stabilisation du 
systeme pour chaque frequence et chaque amplitude et devient 
tres longue; d’ailleurs on tend de plus en plus a etudier non pas 
la reponse du systeme a une sollicitation (frequence amplitude) 
mais la reponse du systeme a une entree non permanente en 
utilisant des criteres statistiques (moyennes). C’est en particulier 
le cas de tous les criteres fondes sur l’erreur quadratiquemoyenne 
minimale. 

s sa ^ a tors Que 1 erreur moyenne depend de la fonction 
d entree. L idee de cette note est de definir des fonctions suscep- 
tibles d etre injectees dans le systeme de faqon aussi proche que 
possible des fonctions reelles auxquelles le systeme devra repon- 
dre, et de calculer l’erreur quadratique moyenne a partir de ces 
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fonctions. Les fonctions qui sont definies ci-dessous peuvent 
etre ajustees, dans line certaine mesure, a des phenomenes 
reels. En particulier Pacceleration maximale d’un signal reel est 
toujours connue (ou tout au moins une borne super ieure, qu’on 
peut en general determiner sans trop de surabondance). C’est 
l’un des parametres principaux dont on dispose dans la defini¬ 
tion du processus generateur. II est evident que si les proprietes 
statistiques de cette fonction aleatoire peuvent etre connues 
mathematiquement, a priori , Pajustement de la fonction alea¬ 
toire au phenomene reel sera d’autant plus simple. 

On dispose done d’un processus generateur permettant de 
definir une fonction aleatoire dont les proprietes mathematiques 
sont connues; l’etude du systeme peut se faire en injectant a 
Pentree une telle fonction (puisqu’on la sait suffisamment 
proche de la fonction reelle); on peut mesurer ou calculer 
Perreur quadratique moyenne entre Pentree et la sortie du 
systeme. 

Conditions a satisfaire 

La condition principale concerne Pacceleration de la courbe. 
Afin de pouvoir calculer les principales proprietes de la fonction 
on definit d’abord une fonction en creneaux dont les instants 
de sauts suivent une loi de Poisson. II est evident qu’on peut 
prendre toute autre loi pour ddterminer ces sauts; cependant 
la loi de Poisson s’adapte d’emblde relativement bien a un grand 
nombre de phenomenes physiques. 



La condition d’acceldration maximale sera defime comme 
suit. Soit ti un instant auquel un saut se produit; Pamplitude du 
saut sera proportionnelle au carr£ de Pintervalle A4- = 4+i ” U* 
Si k est Pacceleration maximale imposee, le coefficient de pro- 
portionnalitd sera dgal k kjl {Figure 1). 

Cette condition implique que la succession des instants ti 
soit connue au moins un pas en avance. Cette condition ne 


presente pratiquement pas de difficultes (on notera cependant 
que Pintervalle separant 2 instants peut, dans une loi de Poisson, 
atteindre n’importe quelle valeur, ce qui necessiterait un dis- 
positif a memoire a duree illimitee). Mais il s’agit ici d’un 
probleme physique et, si Pon admet qu’une probabilite de 10” 8 
par exemple, est negligeable, il suffira de prendre comme duree 
maximale de la memoire, la duree de Pintervalle A 4 correspon- 
dant a cette probabilite; Putilisation d’une loi telle que la loi 
de Poisson rend aisee cette determination. 


Essais preliminaires 


Fonctions a acceleration toujours maximale 

Dans les premiers essais que nous avons fait on s’imposait 
a chaque saut une acceleration maximale, mais le signe de cette 
acceleration etait tire au sort. La fonction primaire en creneaux 
correspondante tend a diverger en amplitude (resultat classique, 
l’ecart quadratique moyen est proportionnel a la racine carree 
du nombre de sauts). Il est possible de ramener la fonction vers 
Paxe, e’est-a-dire, de diminuer le coefficient de proportionalite, 
en imposant un tirage au sort du signe de Pacceleration non 
equiprobable. Par exemple, saut negatif avec une probabilite de 
60 pour cent si la derniere valeur en amplitude est positive. Des 
calculs d’echantillons sur machines a calculer ont montre 
qu’effectivement la courbe se stabilisait de faqon satisfaisante. La 
Figure 2 represente un echantillon repondant aux conditions 
suivantes: acceleration toujours maximale en module; tirage 
40—60 pour cent-—la courbe en pointille correspond a un tirage 
equiprobable. Un tel processus a l’avantage de la simplicite mais 
il est difficile de connaitre les proprietes mathematiques statis¬ 
tiques de cette fonction aleatoire. La determination de la fonction 
de distribution conduit a des calculs inextricables... On peut 
cependant ajuster dans une certaine mesure les parametres 
du processus dans le cas ou les instants t { satisfont une distribu¬ 
tion de Poisson, on dispose: 

{a) du nombre moyen a de points par unite de temps 
(seconde) — alors la probabilite d’avoir n points dans un inter¬ 
vals de temps At est donnee par: 


p(n, Af) = 


e~ aAt (aAt) n 

nl 


(Z>) du parametre de proportionalite k definissant a la fois 
Pamplitude du saut et Pacceleration maximale imposee — si la 
fonction represente des volts k s’exprime en volt [s 2 . 



Figure 2 
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(c) la probability p definissant le rappel vers Laxe, et par 
consequent Techelle 5 de la stationarite et la dispersion des 
amplitudes de la fonction. 

Le parametre a est fortement lie a la notion de frequences 
contenues dans l’epreuve de la fonction; le parametre k, une 
fois fixe a, definit le domaine de variation (en probabilite) de la 
fonction; le parametre p permet d’obtenir la stationarite plus 
ou moins rapidement et fixe la dispersion desiree des amplitudes. 
Bien entendu, on peut utiliser toute loi de distribution des 
instants /$, y compris des lois empiriques. 

Fonctions a accelerations bornees mats non necessairement 
toujours maximales 

(a) Si, au lieu de tirer au sort l’un des deux points 
Pi ± k!2 (At *) 2 on fait un tirage, suivant une certaine loi, sur 
Pinter valle k (At ,) 2 centre sur y € on obtiendra une fonction a 
acceleration quelconque mais bornee. La Figure 3 represente un 
echantillon pour lequel on ferait un tirage equiprobable sur AB. 


suivantes: les changements d’etats de la fonction en crenaux 
suivent une loi de Poisson (nombre moyen de changements par 
unite de temps. 1 ) bornes d amplitude* Af = -4- 5 . L’inconvenient 
de tous ces processus est de ne pas permettre l’ajustement de 
la density de probabilite des ordonnees; le calcul de cette density 
est tres difficile sinon impossible comme deja signale. Nous 
proposons done le processus generateur suivant. 
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Figure 3 



Ce processus n’empeche pas la divergence en valeur quadra- 
tique moyenne deja signalee. On peut alors stabiliser la courbe 
autour de l’axe, par exemple de la faqon suivante: on pondere 
la loi de probability centree sur la valeur a t{~ (cf. Figure 4; 
on suppose toujours que le tirage est equiprobable sur AB) par 
une loi de probability centree sur Vaxe des temps (par exemple 
une loi gaussienne comme indique sur la Figure 4). Le tirage au 
sort s’effectue alors suivant une loi de probability, variable a 
chaque saut (courbe en trait mixte). 

Une variante, plus brutale, consiste a prendre pour loi fixe, 
centree sur l’axe des t une density constante de — M a + M, 
M etant la borne d’amplitude imposee pour la fonction. Le 
processus est alors tres simple, puisqu’il consiste a remplacerle 
tirage au sort equiprobable sur AB par un tirage equiprobable 
sur A'B (Figure 5). 

La Figure 6 represente un echantillon de lOOsauts environ 
lisse, obtenu sur machine CAB 500 repondant aux conditions 


Generalites sur le processus cherche 

Comme indique au premier paragraphe, nous cherchons une 
fonction aleatoire ayant une acceleration maximale bornee en 
module. Nous n’imposons cependant pas que la fonction soit 
toujours a acceleration maximale; Facceleration a chaque 
instant peut avoir une valeur comprise entre les 2 bornes 
(positive ^et negative). On verra plus loin que cette condition 
permet d ajuster la fonction sur une loi de probability en ampli¬ 
tude donnee. 

Fonction generatrice primaire (fonction en creneaux) 

On definit d abord une fonction en creneaux, les instants 
de saut etant definis, par exemple, par une loi de Poisson. On 
admet que la density de probability en amplitude est imposee; 
soit G (Y) cette fonction (Figure 7). Supposons qu’a Tinstant 
considere 1 echantillon de la fonction aleatoire soit tel qu’il 
satisfasse a une loi de densite de probability en amplitude 
constante: soit F 0 (Y) cette loi. La difference G(Y)~ F 0 (Y) 
est certainement positive dans un ou plusieurs intervalles 
de (Y): (Y x> Y ? ) et (L 3 , 7 4 ) sur la Figure 7 (b). 

Soit ti un instant auquel un saut doit se produire et imme- 
diatement apres que rechantillon calcule jusqu’a cet instant ait 
pour densite de probability la densite constante F 0 (Y). Nous 
voulons maintenant que la suite de rechantillon tende vers une 
loi de probability G(Y). A l’instant t t nous ouvrons une premiere 
porte (voir Figure 7 (a)) d’amplitude totale AB = k (t i+1 - 4.) 2 
centree sur la derniere ordonnee a ( 4 - — 0 ); k designe l’accelera- 
tion maximale imposee (module). 

On cherche 1 intersection de cette porte et des intervalles ou 
la difference G (Y) — F 0 (Y) est positive. On fait un tirage au 
sort, par exemple equiprobable, dans les intervalles communs 
a ces deux domaines (intervalles (£, 7 2 ) et ( 7 3 , A) sur la Figure 7). 
On obtient ainsi la valeur du creneau suivant. 
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On continue cette mdthode pendant un certain nombre de 
sauts (par exemple 1000 sauts) puis on calcule la densite de 
probabilite en amplitude de Fechantillon ainsi construit, soit 

(Y) cet dchantillon. Par rapport a la fonction initiale F 0 (Y ), 
F 1 (Y) s’est rapproche de G (Y) (Figure 7 (d)). 

On recommence le mSme processus que ci-dessus mais en 
effectuant maintenant la soustraction G (Y) - F x (Y) et en ne 
retenant que les intervalles oil cette difference est positive. On 
garde ce ou ces intervalles (il y en a 3 sur la Figure 7(d)) durant 
un prochain lot de sauts (1000 par exemple). Apres quoi on 
calcule la nouvclle loi de probabilite F 2 (Y) qu’on gardera 
durant le prochain lot de sauts. 

On peut ainsi esperer tendre apres un certain nombre de 
lots de sauts, vers un processus stationnaire. II est evident, en 
effet, que les premiers sous-dchantillons ne sont pas stationnaires 
puisque, par leur construction meme,ils evoluent lentement en 
density de probability vers la loi imposee G (Y). 

Lorsque la fonction F n (Y) est tres voisine de G (Y) on notera 
qu’il existe de toutes fagons des intervalles ou G(Y) - F n (Y) 
est positif. Cependant, afin d’obtenir une stationnarite plus fine, 
il peut etre avantageux de reduire la periode de mise a jour de la 
fonction de densite de probabilite en amplitude (par exemple 
100 sauts). Ce point ne constitue pas un obstacle a ce que nous 
recherchons: il s’agit d’une fonction aleatoire stationnaire 
d’aspect physique. 

La stationnaritd ne saurait etre obtenue dans un intervalle 
au moins egal a une certaine duree: reduire cet intervalle 
reviendrait a restreindre le caractere aleatoire de la fonction. 

La fonction en creneaux doit etre lissee par des arcs de 
paraboles (Figure 8). Chaque saut est remplace par deux arcs 
de paraboles egaux mais de concavite opposee. Ils se raccordent 
au milieu de Fintervalle 1 “ U • L’ acceleration reste done 
constante de U a (t i+1 - U)\2 et prend une valeur opposee dans 
le demi intervalle suivant. Un tel lissage ne permet pas de passer 
simplement des proprietes statistiques de la fonction en creneaux 
a cel les de la fonction lissee, mais par contre on est assure que 


Facceleration de la fonction ne depassera jamais la borne 
maximale imposee a quelqu’instant qu’on se place. 

On peut cependant dans le cas present calculer mathemati- 
quement la densite de probabilite des amplitudes de la nouvelle 
fonction. 



Resultats 

Donnees 

La mise au point du processus a ete faite sur une machine 
SEA-CAB 500 de l’ENSA. Le calcul de la fonction definitive 
a ete execute par le laboratoire de calcul de l’institut Poly- 
technique de Grenoble* sur une machine BULL-Gamma III E.T. 
Le processus utilise est celui decrit ci-dessus utilisant une suite 
poissonienne de points en abscisse. (Les proprietes statistiques 
de cette suite ont ete verifiees.) 

La courbe de densite d’amplitudes a ete remise a jour tous 
les 250 points et elle a ete imprimee tous les 1000 points. Comme 
indique au § 5 on suppose, au debut du calcul, que la densite de 
probabilite de la Tonction passee’ est uniforme. On fait le tirage 
au sort (equiprobable) dans Fintervalle intersection de (a) l’inter- 
vaile y { ± (A tJ2)\ (b) Fintervalle ± A, A = 5. (les amplitudes 
ont ete quantifiees en 32 niveaux), et (c) les intervalles ou la 

* Nous remercions Mr le Directeur du Service Technique Aero- 
nautique qui a bien voulu supporter les frais de ces calculs, Mr le 
Professeur Kuntzmann et Mr Belino, respectivement Directeur et 
Assistant du Laboratoire de Calcul de Grenoble pour leur collabora¬ 
tion a cette etude. 
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difference: densite de probability imposee moins densite de 
probability actuelle est negative. 

Pour ce tirage au sort on a utilise une table de nombres au 
hasard equiprobables (proprietes statistiques vendees). La 
densite de probability evolue vers la courbe de densite imposee 
(Figure 9). 

Les 5000 derniers sauts pour lesquels la courbe de densite 
est tres voisine de la courbe imposee, ont ete sortis sur cartes 
puis lisses par des arcs de paraboles (pas 0,25). La courbe finale 
a ete sortie sur cartes afin d’en calculer la fonction d’auto- 
correlation. 



Resultats numeriques 

La Figure 10 montre la partie de la fonction finale (lissee) 
comprise entre les pointsf t = 11 000 et t = 11 150 (lissage au 
pas de 0,25). Les Figure 11(a) et 11(b) montrent revolution de la 
courbe de densite de probability des amplitudes: courbe (a) 
apres 1 000 sauts; courbe (b) apres 5 000 sauts; courbe (c) apres 
10 000 sauts, et courbe (d) apres 15 000 sauts. 

La Figure 12 represente la fonction d’autocorrelation portant 
sur un echantillon de 4 000 points de la fonction lissee ; l’echan- 
tillon utilise a ete preleve dans les 5 000 derniers sauts calcules 
avant lissage soit 20 000 points apres lissage. Pour le calcul de 
la fonction d’autocorrelation on a utilise des sous-echantillons 
de 4000 points. La Figure 12 correspond au sous-echantillon 
compris entre les 4 000 e et 8 000 e points. 

Calcul direct des proprietes statistique 

Puisque le density de probability constitue une donnee du 
probleme, la propriety statistique la plus interessante a deter¬ 
miner est la fonction d’autocorrelation ou le spectre. Nous avons 
aborde ces calculs mais ils sont inextricables dans toute leur 
generality. Aussi avons-nous recherche une expression approchee 
de la fonction d’autocorrelation en procedant ainsi. 

L’approximation consiste a ne calculer que les termes appor- 
tantune contribution importante dans la fonction d’autocorrela- 

t On appelle e saut’ les instants oh la fonction en crenaux subit une 
discontinuity et ‘points’ les abscisses ou l’ordonnye de la fonction 
lissee a ete calculee. (II y a 4 points par saut.) 
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tion: ce sont les termes correspondant a de petits decal ages. On 
note, en effet, sur la Figure 12 que la correlation tombe en dessous 
de 0,15 pour un decalage inferieur a 20 sauts. On envisage alors 
les differents cas possibles: pas de saut dans 1’intervalle t, un 
saut, deux sauts, etc. ..., on calcule les correlations dans chacun 



des cas et on les affecte de la probabilite de realisation de chacun 
de ces cas. Ces probabilites sont connues et simples dans le cas 
d’une loi de Poisson. La valeur de r est elle-m6me liee a l’echelle 
des temps adoptee: si on a pris A sauts, en moyenne, par seconde 
la durde moyenne d’un saut est 1/A seconde et r doit etre 
surtout etudie dans l’intervalle 0,05 1/A et 5/A. Alors le nombre 


de termes intervenant dans la fonction d’autocorrelation est 
limite. L’expression complete est cependant compliquee. 

Conclusions 

Le processus generateur de la fonction aleatoire suggere 
permet d’obtenir des echantillons ayant les proprietes suivantes. 
(a) l’acceleration est bornee en module — l’acceleration peut 
etre toujours maximale ou peut varier entre les deux bornes 
(positive et negative); ( b ) la densite de probabilite des amplitu¬ 
des est imposee; (c) 1’echelle des temps est ajustable par la loi 
de probabilite de la fonction gdneratrice primaire (crenaux). Si on 
utilise une loi de Poisson on dispose du nombre moyen de sauts 
par unite de temps. ( d ) elle est facilement reproductible par 
machine a calculer meme en ‘temps reel’. 

L’utilisation de ces fonctions semble particulierement 
indiquee en Automatique pour la recherche d’optimums soit 
par voie experimentale (on injecte une telle fonction et on 
effectue le reglage des parametres avant la mise en service du 
systeme) soit par voie semi experimentale. Une fois les principales 
proprietes de la fonction estimees (amplitude, acceleration, 
echelle des temps, densite de probabilite) on peut calculer un 
echantillon puis sa fonction d’autocorrelation, puis son spectre 
et accessoirement d’autre proprietes statistiques. 


DISCUSSION 


F. Mesch, Institut fur Regelungstechnik , TH Darmstadt , Germany 

At the beginning of the paper it is stated that the object was to generate 
a process with adjustable amplitude density, adjustable maximum 
acceleration and adjustable power spectrum. I would like to ask how 
far it was possible to adjust the amplitude density and the power 
spectrum independently, since the relationship between these is 
known to be very involved. 


M. J. Pelegrin, in reply 

The adjustment I referred to concerned a limit of the acceleration but 
not the acceleration spectrum itself. This one is directly related to the 
amplitude spectrum; this is the second derivative of the autocorrelation 
function or the product by ( jco ) 2 of the amplitude spectrum for a 
stationary random process. Limitations of the modulus of the accelera¬ 
tion give very loose restrictions on the amplitude spectrum (particu¬ 
larly in the high frequencies). However, this compatibility is the 
result of the physical aspect that we are looking for. 
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Summary 

A relation is derived by means of which the uncertainty of an arbitrary 
parameter pertaining to a linear mathematical model can be calculated. 
It is assumed that the model is identified with the process, industrial 
or other, on the basis of finite time terminal data records, according 
to a least mean square error criterion. The process is assumed to con¬ 
tain stationary noise sources distorting the output record. 

The uncertainty relation is valid for deterministic and stochastic 
input signals to the process and for all measuring and evaluation 
procedures that satisfy the least mean square error criterion. Well- 
known procedures of that kind are the conventional frequency response 
method and the transfer function determination by the Wiener-Hopf 
integral equation on the basis of statistical data. 

The parameter error R.M.S. value, or the parameter uncertainty, is 
related to the following factors: the process input signal and the process 
noise, represented by their power density spectra, the time period of 
measuring and a parameter influence function. 

The information disclosed by the uncertainty relation is dealt with 
and two examples given. 

Sommaire 

On a obtenu une relation permettant de calculer Fincertitude d’un 
parametre arbitraire dans un modele mathematique lineaire. Ce der¬ 
nier, represente un processus, industriel ou non, a partir des donnees 
enregistrees durant un intervalle de temps fini, et depouillees selon le 
critere des moindres carres. On suppose que le processus contient des 
sources de bruit stationnaires deformant le signal de sortie enregistre. 

La relation obtenue est valable pour des signaux deterministes et 
stochastiques et pour toutes les procedures de mesure ou devaluation 
utilisant le critere des moindres carres. Parmi ces procedures, se 
trouvent la methode frequencielle classique et celle de determination 
des fonctions de transfert a Faide de l’equation integrate de 
Wiener-Hopf sur la base de donnees statistiques. 

Les facteurs influenqant la valeur de Fincertitude sont: le signal 
d’entrde et le bruit du processus, reprdsentes par leur densite spectrale, 
la duree de la mesure et une fonction d’influence parametrique. 

Le contenu informationnel de la relation est discute. Deux exem- 
ples sont donnes. 

Zusammenfassung 

Mit Hilfe der hier abgeleiteten Beziehung kann man die Unbestimmt- 
heit eines beliebigen Parameters in einem linearen mathematischen 
Modell berechnen. Es wird angenommen, daft die Registrierung der 
Ein- und Ausgangsgrofle fiber ein endliches Zeitintervall die Annahme 
des Modells ffir den (industriellen Oder anderen) ProzeB rechtfertigt, 
wobei als Kriterium der kleinste quadratische Mittelwert des Fehlers 
zugrunde liegt. Der ProzeB enthalte Quellen ffir stationares Gerausch, 
das die Registrierung der AusgangsgroBe verfalscht. 

Die Unbestimmtheitsrelation gilt ffir deterministische und stocha- 
stische EingangsgroBen und fur alle MeB- und Auswerteverfahren, die 
dem Kriterium des kleinsten quadratischen Mittelwertes des Fehlers 
genfigen. Wohlbekannte Verfahren dieser Art sind das fibliche Fre- 
quenzgangverfahren und die Bestimmung der Ubertragungsfunktion 
durch die'Wiener-Hopfsche Integralgleichung auf Grund statistischer 
Daten. 

Die Unbestimmtheit des Parameters (Effektivwert des Fehlers) 
steht mit den folgenden GroBen in Beziehung: dem Eingangssignal 


und dem Rauschen des Prozesses, die durch das Leistungsspektrum 
dargestellt sind, der MeBzeit und einer ParametereinfluBfunktion. 

Die durch die Unbestimmtheitsrelation gewonnene Erkenntnis 
wird erlautert; zwei Beispiele sind angeffihrt. 


Introduction 

The background of the work to be presented is the problem of 
identifying an industrial or other process, and a mathematical 
model, in order to prepare for the design or the adjustment of 
a control system regulating the process. The information 
available about the process transfer characteristics is assumed 
to be a finite time record of the process input and output 
variables. It is further assumed that the output is distorted by 
a noise component, added by sources within the process or in 
the measuring device. 

A linear mathematical model is used to represent the pro¬ 
cess, which therefore should be at least approximately linear. 
The model is specified by a number of parameters; these are to 
be adjusted so that the model, subject to the actual input signal, 
will produce an output as similar as possible to the real process 
output. A least mean square error criterion is used here—this 
implies that the model shall not contain noise sources, because 
such sources, if uncorrelated to the corresponding sources of the 
real process, will on average only increase the mean square 
value of the output difference. 

The best possible model is obtained after a proper adjust¬ 
ment of the model parameters. This model will not be exact, 
however, because the process noise and the finite observation 
time will make the parameter values uncertain, to some 
degree. 

In this paper the uncertainty of an arbitrary parameter is 
estimated and general expressions given, relating the parameter 
uncertainty to important factors of the model design problem. 
Those expressions may be used to answer questions such as: 
for how long is it necessary to measure to allow a specified 
model accuracy? How complex and detailed can the model be 
made on the basis of a given amount of experimental data? 
What kind of input signal will give the least uncertainty of the 
parameter values ? 

Regarding the nature of the input signal, no specific as¬ 
sumption is necessary at this stage. Being recorded and thus 
completely known, it is of no importance if the input signal 
was generated by a deterministic or a stochastic source. It is 
shown later that a record of the input signal is not needed in 
the stochastic case, provided the statistical properties of the 
signal source are known. 

The first step towards the parameter uncertainty is a com¬ 
putation of the parameter deviation in a single test, under the 
auxiliary assumption that the process noise component is ex¬ 
plicitly known. In the second step, the average square value of 
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the said deviation is derived for a large number of tests. In 
this procedure the explicit noise signal is replaced by a statistical 
function characterizing the noise source. The R.M.S. value of 
the parameter deviation is considered as a measure of the para¬ 
meter uncertainty. 


The Parameter Deviation in a Single Test 

The difference e(t) between the process and the model output 
signals is by definition: 

e(t) = y(t)+n(t)-q(t) (1) 

The mean square value of the difference can be written as 
follows: 

e 2 = y 2 + n 2 + q 2 + 2 yn - 2 yq — 2 nq (2) 


The bar indicates the mean value over the period of time 
during which the signals have been recorded. 

The model is adjusted by means of N parameters a t in such 
a way that the mean square value of eqn (2) will arrive at a 
minimum. The signal q(t) is also a function of the parameters 
a it and the differentiation with respect to these leads to the 
following N equations: 


or 


dg2 _o M 
da ; 9 0a, 


o a t ocii 


( 3 ) 


(q-y) 


dq_ 

da t 



(4) 


It is now assumed that the set of parameters is chosen in such 
a way that the model and the process are identical for a certain 
set of parameter values, denoted by a i0 . Due to the process 
noise and the limited time of measurement, the parameter values 
actually obtained in the minimizing procedure will deviate from 
the values a i0 . At the same time, the measurement, if serving any 
purpose, has to be so good that the deviations are small. A 
Taylor series expansion is then possible. 


00 1 

q-y= E y\ 

k= l K ' 


N 

E da s 

L/-i 



( 5 ) 


Under the assumption that terms of an order higher than 
one can be neglected, eqn (5) is introduced into eqn (4). 


JV 


E da j 

i = i 





( 6 ) 


The parameter deviation of interest may be obtained by the 
solution of the N linear, simultaneous eqns (6). Further restric¬ 
tions will, however, be introduced with respect to the set of 
parameters in order to make possible the desired result. That is, 
a formula, valid for a single arbitrary parameter of interest, 
relating the parameter deviation to the most important factors 
of the problem; such as the necessary time of recording, the 
nature of the parameter and so on. Eqn (6) requires that a com¬ 
plete set of parameters must be defined before the probable 
deviation of any single parameter can be estimated; this is 
because there is generally a coupling effect between the different 
parameters. 

In the following, only one parameter will be considered—that 


is the interesting one, denoted by The choice o is P aia 
meter will be unrestricted. The other parameters, mvo vec in e 
same minimizing procedure, will be defined to fulfil the o owing 
conditions of orthogonality. 



j^i 


( 7 ) 


The orthogonal parameters can be constructed as lineai 
combinations of the original parameters plus extra, auxiliary 
parameters. The volume of the computation will increase by the 
rearrangement of the parameters and by the fact that the mini¬ 
mizing procedure has to be repeated for every parameter in¬ 
vestigated. This is of no importance, because the purpose here 
is to estimate the parameter errors only, and it is not necessary 
actually to perform the computations. 

Taking eqn (7) into account and neglecting the unimportant 
difference between the partial derivatives at the points a; and a iQ 
the parameter deviation can be written as follows: 


da ; — 



( 8 ) 


After introduction of the model weighting function the 
following alternative expression for the parameter deviation is 
obtained. 


da, = - 


d u ( n (0 x (f — u) 

/ego __ 


du 


d / se (tf)\ f QQ ( V ) 

EA da i 


x(t~~~u)x{t — v) 

ai ° ( 9 ) 


The expression eqn (9) relates a certain parameter deviation 
to properties of the model weighting function, the input signal 
and the process noise. The time interval T ni of the test is inherent 
in the averaging procedures indicated by the bars. The expression 
is valid for a parameter isolated by the requirements of ortho¬ 
gonality. 


Estimation of the Parameter Deviation R.M.S. Value 


A stationary process noise source with a zero mean output 
makes the average deviation da t zero, because positive and 
negative outcomes of the same magnitude are equally likely. 
Considering the mean square value, it is convenient to treat the 
numerator (r) and the denominator ( d ) of eqn (9) separately, 
the former being a stochastic quantity and the latter a de¬ 
terministic one. 

The auxiliary function p(t) allows us to write the numerator r 
as follows: 


p( o= 






( 10 ) 


r = p(t)n(t) (ID 

In the next step, the numerator will be squared and the bars 
exchanged by the corresponding integral signs. Simultaneously 
a time displacement v of the noise signal will be introduced as 
a tool to shift any interval of the endless stream from the noise 
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source into the range of measurement T m . An index p is at¬ 
tached to the symbol r to indicate that the result is valid for a 
certain function p{t). 


-2 / . 1 


M= 


TJ 


dh 


dt 2 p (fj) p (f 2 ) n(v+t 1 )n(v+t 2 )( 12) 


The average, with respect to v 3 of the squared numerator is 
obtained if the noise signals under the integral signs are ex¬ 
changed by the noise auto-correlation function (p nn (t 2 — 4 ). 
For convenience the difference t 2 — t x is also replaced by t 
giving the following result. 


r P=i dt d *iP(*i)p(*i + 0]<P„ n (0 (13) 

x m J — 00 LJ — 00 

The function within the brackets is also an auto-correlation 
function and may be denoted by <p 9S >(t). This correlation should 
be computed on the basis of the recorded data about x(t). In 
the case, however, where the input signal is generated by a 
stochastic source the correlation function might be computed 
directly from the source characteristics. It is therefore sufficient 
to know the input signal in statistical terms only. The mean 
square value of the numerator r can therefore be written as 
follows, both for deterministic and stochastic input signals. 

_ if 00 

r 2 =— dup pp (t)cp nn (t) (14) 

J - 00 

Often the signal power density spectrum, $ xx (co), can pref¬ 
erably be used in the calculation of the integral eqn (14). Ac¬ 
cording to eqn (10) we have: 


# w (®)=#»(®) 


3 g(« ) 


(15) 


Parseval’s formula applied to eqn (14) gives then the follow¬ 
ing alternative expression for the numerator mean square value. 




2 71 

T~ 

x m 


d<o$ xx (co)<t> nn (co) 


dQXjco ) 
da t 


(16) 


The denominator d of the eqn (9) is independent of the 
noise and has a definite value when the input signal x(f) is 
given. This value is equal to the mean square value of pit), as 
can be seen by comparing the denominator and the definition 
ofp(t), eqn (10). - T — 

d =r(0 (17) 


In the case of stochastic input signals, use can be made of 
the eqn (15) giving this alternative expression for the de¬ 
nominator. 


d =<Ppp( 0 )= f 

J — 00 


dco <P xx (co) 


dQ (jco) \ 2 
. da, J ai0 


(18) 


Finally we are able to express the parameter deviation R.M.S. 
value, or the parameter uncertainty, in one single relation by 
the introduction of eqns (16) and (18) into eqn (9). 



I: 

dco $ xx (co) <f>„„ (m) 

00 

fdQ(jco)\ 

V da, J 

2 


"* 00 

dm $ xx (co) 

J — CO 

( Q Q(jco)\ 

V L o 

2 ”l: 

l 


(19) 


The uncertainty relation is here presented in the frequency 
domain. It can readily be written also in the time domain by use 
of eqns (14) and (17). The relation will then, however, be some¬ 
what more complicated due to the two convolution integrals 
concealed in Numerical and graphical calculations seem, 

in general, to be easier to perform in the frequency domain. 


First Example: A Frequency Response Test 

In the first example, a sinusoidal input signal, a sin vt, is 
used to determine the three parameters K, T and £ of a simple 
process, described by the following transfer function: 


G(jco) = 


K t~ jmL 
1 + jcoT 


( 20 ) 


For a numerical illustration these specific values of the 
parameters, K = 1, L = Fare chosen. 

The noise power density spectrum is assumed to be constant 
over the frequency range of interest. 


®nn(o>)=N 0 unit 2 /rad/sec (21) 


iV 0 = 


a 2 T 

400 


( 22 ) 


The noise power of eqn (22) corresponds to a ratio 1/10 
between the R.M.S. value of noise components within the fre¬ 
quency band -1/T to +1/T and the R.M.S. value of the 
sinusoidal input. 

From eqn (20) the following partial derivatives are obtained. 


G K (jco) = 


e~ JoL 

1+jcoT 


G r (jco) = 


- jcoKe~ JcoL 
(l + jcoT) 2 ~ 


g l 0 '®) = 


— jcoKe~ JaL 
1 + jcoT 


(23) 


The uncertainty relation, eqn (19), is now simplified by the 
facts that the noise spectrum is a constant and the area of the 
input signal spectrum is concentrated at the frequencies + v. 
The remaining integral to be solved, the value of which is given 
by the definition of the spectrum, is as follows: 

oo 2 

d (oO xx {co)=~ (24) 

J ~ 00 

The substitution of eqns (21), (23) and (24) into the relation 
eqn (19) yields the following result: 


Ik 1 

K 2 

II 

N 0 
KFP 

(i + 

v 2 T 2 ) 

(25 a) 

df 1 

47T 

N 0 

/_ 

1 \ 2 


-fr- 

~Y~ 

x m 

K 2 a 2 


(25b) 

dU 

471 

N 0 I 

( f\ 

2 1 +v 2 T 2 

(25c) 
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A low frequency is preferable in the determination of the 
gain K, a high frequency better for the lag L, and the specific 
frequency v = 1/T best with respect to the time constant T. 
Choosing the latter frequency and introducing the numerical 
data given above one gets 


(d K 2 f 
K 


= 0-25 



(d T 2 f 
T 



(d L 2 f 
L 



(26) 


To determine the parameters K, T, and L within a relative 
accuracy of 5 per cent, the measuring time T m should be at least 
25 T\ 50 T and 25 T respectively. 

The result, eqn (26), is independent of the evaluation proce¬ 
dure, provided an optimal one is employed. In one conventional 
method of frequency response testing the output is multiplied 
by a sine and a cosine function and, after filtering, the ‘in-phase’ 
and ‘quadrature’ components are recorded. This method is 
optimal in the sense assumed here. A direct calculation of the 
parameter errors, when the parameters are derived in the best 
possible way from the in-phase and quadrature components, 
will give exactly the result of eqns (25) and (26). The calculation 
is too lengthy to be included in this paper. Reference is made in¬ 
stead to the well-known fact that the conventional frequency 
response method is a kind of Fourier transformation, which can 
be developed from the least mean square error criterion being 
our starting point. 


Second Example: A Random Input Signal 


main under the integral sign when eqn (23) is substituted for 
eqn (19). This implies, however, only an unimportant complica¬ 
tion, as the integral can be solved in a straightforward way by 
contour integration (if not found in a list of definite integrals). 

The uncertainty relations will take the following form after 
the integrations have been carried out. 

cLK^_2ti N 0 T 0 /T 

F'r M xV ro/r _i +e -v 

df T _47T No TJT 

T 2 T m K 2 a 2 1-(1 + T 0 /T)&~ To,t 1 J 

ID _2n N 0 f T\ 2 T 0 /T 
L 2 ~T m K 2 a 2 \Lj l - C - T ° IT 


A long time interval T 0 favours the determination of the 
gain K , the shortest possible interval will suit the parameter L 
and a certain interval T 0 — 1,8 Tis the best choice with respect 
to the time constant T. This value of the interval will be accepted 
for the following numerical example. 

As in the first example a ratio of 1/10 will be assumed be¬ 
tween the R.M.S. value of the noise components in the band 
— 1/T to +1/7’ and the input signal R.M.S. value. 


N 0 = 


a 2 T 

200 


(29) 


The introduction of eqn (29) and the values TJT — 1,8, 
L = T and K = 1 into eqn (28) will give this result. 


(d K 2 f 

K 



The same process, eqn (20), is studied in this example but 
this time a random binary input signal is employed. The input is 
constant + a or — a during successive and equal intervals T 0 , 
the probability of each sign being 1/2. 

The evaluation of the parameters K , T and L can be done 
according to the scheme indicated in the introduction; a model 
adjustment by means of the recorded input and output signals. 

The process transfer function can also’ be obtained via the 
Wiener-Hopf integral equation, if appropriate correlation func¬ 
tions are computed by use of the recorded data. The solution of 
the integral equation fulfils by definition the same least mean 
square error criterion used in this paper. The uncertainty re¬ 
lation, eqn (19), should therefore be applicable. The deduction 
of this equation was, in fact, in the beginning of this work, based 
upon the Wiener-Hopf equation, before a direct approach was 
found preferable. 

The auto-correlation function <p xx ( t) of the binary signal 
can be found in several textbooks and the power density spec¬ 
trum is readily obtained by Fourier transformation. 


( Pxx( z )~a 2 



$xx (<») = 


a 1 — cos co T 0 


nT 0 


co 


(27) 


In the present example, the spectrum of the input is dis¬ 
tributed and the partial derivative functions, eqn (23), will re- 


(dT 2 )* 

T 


= 0-46 



L 


=0-26 



(30) 


The measuring time T m to get a relative accuracy of 5 per 
cent should here be at least 23-57", 85 T and 27 T for K, T and 
L respectively. 

The parameter uncertainties of the stochastic case, eqn (30), 
can directly be compared with the result of the preceding ex¬ 
ample of a deterministic input signal. The general conditions 
are the same in both cases. No discrepancy can be observed 
with respect to the result. 


Conclusion 

The derivation of this paper resulted in a relationship be¬ 
tween the uncertainty of an arbitrary model parameter and four 
important factors of the identification problem. These factors 
are: the input signal and the process noise, represented by their 
power density spectra, the time period of measuring and the 
influence of the parameter being considered upon the model 
weighting or transfer function. The parameter influence is 
measured by the partial derivative of the latter function with 
respect to the parameter of interest. 
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It is possible to express the uncertainty relation in the time 
or in the frequency domain. The frequency domain presentation, 
eqn (19), describes the main features of the relation in an easier 
way and is more suitable for approximate and graphical com¬ 
putations. 

It will be of interest to sum up the qualities of the factors 
mentioned above, that will tend to reduce the uncertainty 
of the model. 

(a) The input signal should be large, and located as much as 
possible within a frequency range defined by the parameter 
influence function. 

(b) The process noise should be small, and located as much as 
possible outside the frequency range given in (a). 

(c) A long time of measurement is favourable and the uncertainty 
is inversely proportional to the square root of that time. 

Although the present result in the first place is useful in the 
planning of transfer function measurements, it illustrates also a 
dilemma of adaptive control. In such a control system a process 
parameter is continuously evaluated from process input and 
output data to initiate an adjustment of the controller, com¬ 
pensating for parameter variations. The time of measurement is 
equivalent to a time delay in the loop of adaptation; a short 
time delay means therefore a fast, but incorrect, compensation. 
A long time of measurement allows the compensation to be 
correct in size, but too late in time. Thus the uncertainty relation 
indicates that some unfavourable parameter variations cannot 
even theoretically be compensated for by adaptive control. 


Nomenclature 

x(t) 

y(t) + n(t) 
y(t) 


n{t) 


q(t) 


The process (and model) 
input signal 

The process output signal 
The deterministic part of 
the process output signal 
The stochastic part of the 
process output signal 
The model output signal 


«(0 = y(f) + n(t) — q(t) 

The difference between the 
process and the model out¬ 
put signals 

T m 

The period of time during 
which the process input and 
output signals have been 
recorded 


The bar indicates the aver¬ 
age with respect to time 
over an appropriate time 
interval, usually T m 

t, T, U, V 

Time variables 

2(««, r) 

The model weighting func¬ 
tion 

Q(ja>) 

The model transfer function 
or the Fourier transform 

of eco 


The parameter number i of 
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DISCUSSION 


P. Eykhoff, Technological University, Electrical Engineering Depart¬ 
ment, Delft, Netherlands 


The contribution by Dr. Qvarnstrom deserves to be studied by all 
who are concerned with process-parameter estimation. 

The time intervals needed for information processing that result 
from his studies clearly indicate the basic limitations inherent in 
‘tracking 5 of unknown parameters. 

In the paper, use is made of orthogonality, which leads me to the 
following questions: 

(1) In eqn (7) of the paper, a condition of orthogonality is stated. 
For stochastic types of input signals this condition is difficult to 
fulfil, the construction of a set of filters orthogonal with respect to a 
single, time limited element of a stochastic process seems to be highly 
impractical. So the best we can hope for is 


= 0 


(i; 


AD 

\pCL-J fl . \9 ttj)ajo, 
where E [ ] indicates the mathematical expectation. Do you have a 
estimate of how much the results that are derived in the paper degrad 
as a result of this change of orthogonality conditions ? 


(2) Just below eqn (7) of the paper is stated ‘The orthogonal 
parameter can be constructed as linear combinations of the original 
parameters plus extra, auxiliary parameters’. What are the theoretical 
and practical limitations of this procedure with respect to the para¬ 
meter interval over which it holds ? 

(3) In spite of the conditions of orthogonality introduced in the 
derivation of the paper’s eqn (19) one notices in the example given 
that orthogonality of the parameters is not fulfilled. What are the 
implications of this with respect to the answer obtained ? 

(4) On the third page of the paper the idea of an optimal evaluation 
procedure is coined. In engineering terms, what does constitute such 
an optimal procedure ? 

F. B. Tuteur, Department of Engineering and Applied Science, 

Yale University , New Haven, Connecticut, U.S.A. 

The author is to be congratulated for his simple and straightforward 
application of modern decision theory to the problem of system 
identification. The results appear to be quite universal, and they place 
a lower bound on the errors that can be obtained from measurements 
made during a finite time. Thus, they would appear to settle once and 
for all the question of whether it is possible to find an identification 
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scheme that is very much superior to others that have been considered 
in the past. It is interesting to note that the R.M.S. error varies 
inversely with the square root of the estimation time. This is, of 
course, in accordance with typical results in estimation theory and 
serves as a qualitative check. 

Since the author has confined himself to the estimation of a known 
set of parameters, the paper does not consider the problem of iden¬ 
tification of a system whose exact structure is not known. I would, 
however, be interested in the author’s opinion about the possibility 
of applying his method in a situation where the structure is not known, 
or only partially known, i.e. where the system has more or different 
parameters than the model. 

E. Blandhol, Eidanger Salpeterfabriker , Heroya, Porsgrunn, Norway 

Two very important points in the practical use of model methods are 
the choice of model structure and the convergence of the experimental 
procedures. 

The author makes the following two assumptions: (a) that the 
model and process are identical for a certain set of parameters at 0 , 
and ( b ) that the model parameters are orthogonal. Assumption ( a ) 
can only be satisfied if the system structure is exactly known and if 
it is possible to duplicate it in the model. In a mathematical model 
of a ‘black box’ system the assumption will probably be invalid. 
Assumption (b) of orthogonal model parameters may be satisfied in 
a mathematical model, but usually not in a model duplicating a given 
system structure. I would like to ask the author how important the 
two assumptions are for the validity of his results. 

The convergence of the model adjustment depends on the form 
of the ‘performance surface’, and not only in the vicinity of optimum, 
where the author’s formulae are valid, but in a broader range. In the 
paper by J. G. Balchen and myself we observed, by our experiments, 
that for certain model structures and adjustment strategies the method 
may not converge at all. The existence of convergence must therefore 
be known before the author’s results can be applied in a practical 
case to judge the results. 

In his derivations the author introduces the statistical properties 
of the input and the noise, thereby arriving at measurement times that 
seem unnecessarily long. It is quite possible to use very short test 
signals of a simple form, such as a step or ramp, and still obtain useful 
results. In our paper we have studied both first-order and second-order 
systems with output noise, and the true values, with a measuring time 
of only 8 T, where T is the largest time constant in the system. This 
seems to be somewhat better than predicted by the author. 

J. E. Rijnsdorp, KoninklijkelShell Laboratorium, Amsterdam , Nether¬ 
lands 

In your first example you consider a frequency response test. Should 
not the time for completion of the transient response to the sinusoidal 
test signal be added to the measuring time T m ? 

B. Qvarnstrom, in reply 

In his first question Dr. Eykhoff emphasizes a problem arising from 
the fact that the orthogonality condition is dependent on the input 


signal. I am not at present prepared to give a precise and general 
statement regarding the consequences of that dependence, but would 
like to say the following: 

(1) The result is not very sensitive to the effect of coupling between 
different parameters, as demonstrated in my oral presentation. 
Therefore, a moderate lack of orthogonality should not upset the 
result. 

(2) Although it might be possible to construct a case in which the 
input signal in a critical manner will affect the condition of orthogo¬ 
nality, I have not found such a case when a stochastic signal has 
been employed. 

The model adjustment procedure, being the basis of my work and 
of others, can certainly be regarded as a very good one, because all 
information available is utilized in forming the model. Accepting the 
mean square error criterion I consider, by definition, this procedure 
to be an optimal one. There are other procedures giving exactly the 
same parameter uncertainty; these should, therefore, also be optimal. 
One example is the frequency response test, when use is made of the 
Fourier or correlation method. 

Dr. Tuteur and Mr. Blandhol are concerned with the problem of 
processes of unknown structure. Although not clearly stated in my 
paper, it is fairly obvious that the result can be used in the case of such 
processes, provided the following interpretation of the uncertainty 
is made. The formula eqn (19) gives the minimum uncertainty of a 
specific model parameter that can be obtained by the best possible 
use of the measured data. It is not necessary that a corresponding 
process parameter exists. The process or system may therefore have 
more or different parameters than the model. 

If the parameter under consideration is coupled to other para¬ 
meters of the system, this effect will on the average increase the uncer¬ 
tainty above the minimum value. Here we have a choice, orthogonal 
parameters can be used, which is theoretically possible, or the coupling 
can be taken into account by a more complex uncertainty formula. 

Mr. Blandhol brings up the important question about the con¬ 
vergence of the model adjustment procedure. That problem is outside 
my work. When a suitable procedure is found, my formula will 
indicate how much the parameter values obtained are affected by the 
process noise. In one case of interacting parameters I found that the 
procedure did not converge, but at the same time the appropriate 
uncertainty formula (not given in the present paper) indicated infinite 
parameter uncertainties. 

It is not quite correct that statistical properties of the input signal 
are introduced. My result is valid also for deterministic input functions 
and reveals that under favourable circumstances a short measuring 
time is possible. I have checked my uncertainty estimation with the 
result of the paper by Messrs. Blandhol and Balchen and found a 
reasonable agreement. 

Mr. Rijnsdorp is quite right. There might, however, be several 
different situations. The sinusoidal signal might be started and turned 
off within the measuring period, which should then include the 
transients also. In the case of a continuous sinusoidal signal the 
transients will be lost; it is then necessary that the measuring time is 
long compared with the time constants of the process. 
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Summary 

The purpose of this paper is to show how an axiomatic development 
of the theory of control systems can systematize important concepts. 
Starting with the idea of attainable set (the set of points which can 
be reached from a given one in a certain time) a set of 6 axioms is given 
to describe the behaviour of a general control system. Barbashin 
already gave such an axiomatic approach, and Zubov also used an 
axiomatic theory for proving stability theorems. But besides the strong 
form of stability and invariance of a set (given by Zubov) a weak form 
of these properties can be defined for general control systems. In 
general, almost every property of the classical results can be translated 
in a strong and a weak form for control systems. Accordingly, the 
powerful second method of Liapunov can also be applied in a strong 
and a weak form, in order to prove strong and weak properties, 
respectively. 

In this paper is given the set of basic axioms (more or less equiva¬ 
lent to Barbashin’s); some basic results are discussed; the strong and 
weak form of invariance and stability properties and the corre¬ 
sponding strong and weak Liapunov functions are defined, giving the 
statement of the fundamental stability theorems (the detailed proofs 
are given elsewhere). 

Sommaire 

Dans ce travail on montre comment un developpement axiomatique 
permet d’exposer systdmatiquement d’importantes notions de la theorie 
des systemes de commande. Avec la notion d’ensemble accessible 
(Tensemble des points qu’on peut atteindre d’un point donne en un 
certain temps), on peut caract&riser les systemes de controle par 
6 axiomes. Barbashin a donne un systemes d’axiomes analogues et 
Zubov a utilise un systeme axiomatique pour demontrer des theoremes 
concernant la stabilite. Mais parallelement a la forme forte de la 
stabilite et de 1’invariance d’un ensemble, utilisee par Zubov, on peut 
definir une forme faible. En general, la plupart des proprietes classi- 
ques des systemes dynamiques peut etre traduite d’une maniere forte 
ou faible pour les systemes de commande. La seconde methode de 
Liapunov est aussi applicable aux formes forte et faible. 

Dans ce travail on donne un ensemble d’axiomes (semblables a 
celles donnes par Barbashin), les principales consequences sont discu- 
tees, on definit les formes faible et forte de notions des stabilite et 
d ensemble invariant, de plus les fonctions de Liapunov on sense fort 
et faible et l’on enonce les principaux theoremes (les demonstrations 
seront donnees ailleurs). 

Finalement on discute les relations avec des resultats classiques 
(conditions de differenciabilite) et les possibles developpements ulte- 


Zusammenfassung 

Zweck dieses Beitrages ist es, aufzuzeigen, wie eine axiomatische Ent* 
wicklung der Regelungstheorie wichtige Begriffe systematisieren kann. 
Ausgehend vom Begriff der „erreichbaren Menge“ (einer Menge von 
Funkten, die m bestimmter Zeit von einem gegebenen Punkt aus- 
gehend erreicht werden konnen) werden sechs Axiome angegeben, urn 
das Verhalten ernes allgemeinen Regelungssystems zu beschreiben. 
Barbaschin hat bereits einen derartigen axiomatischen Weg angegeben 
und auch Zubow verwendete eine axiomatische Theorie, urn Stabili- 
atstheoreme zu beweisen. Aber neben der strengen Form der Stabi- 
htat und der Invarianz einer Menge (bei Zubow) kann eine schwache 
Form dieser Eigenschaften fur allgemeine Regelungssysteme definiert 


werden. Im allgemeinen laBt sich fast jede Eigenschaft der klassischen 
Ergebnisse fur Regelungssysteme in eine strenge und in eine schwache 
Form ubersetzen. Entsprechend kann auch die wirkungsvolle zweile 
Methode von Ljapunow in einer strengen und in einer schwachen 
Form angewendet werden, um entsprechend „strenge“oder„schwache“ 
Eigenschaften zu beweisen. 

Der Beitrag enthalt die grundlegenden Axiome (sie sind mehr oder 
weniger denen von Barbaschin gleichwertig), die Diskussion einiger 
grundsatzlicher Ergebnisse, die strengen und schwachen Formen der 
Invarianz und der Stabilitatseigenschaften sowie eine Definition der 
zugehorigen strengen und schwachen Ljapunow-Funktionen, welche 
zu den grundsatzlichen Stabilitatstheoremen fuhren. Die ins einzelne 
gehenden Beweise sind anderenorts angegeben. 

SchlieBlich werden der Zusammenhang mit den klassischen Er- 
gebnissen (Bedingungen der Differenzierbarkeit) sowie einige weitere 
Entwicklungen kurz erwahnt. 


Introduction 

From its beginning, the theory of control systems developed as 
a branch of the theory of differential equations. More general 
approaches, including differential difference and integral 
equations, are also important, but the most developed theory is 
related to the classical differential equations. The problems of 
control theory appear, this way, as special applications of this 
theory; but in a certain sense the problem of control theory can 
be regarded as more general than the one corresponding to 
differential equations, as will be seen. Now, the theory of dif¬ 
ferential equations has been successfully developed and gener¬ 
alized to the theory of dynamical system 2 ’ 3 . Instead of starting 
with the equations, one assumes a set of axioms which 
characterize the most basic properties of the solution curves, 
obtaining all the desired results directly from those axioms. 
The advantage is that, in this way, the knowledge and even 
the existence of the differential equations governing the motion, 
is not assumed, so that, for example, the differentiability con¬ 
ditions for the solutions can be relaxed. There is a temptation, 
therefore, to develop the theory of control systems along the 
same lines. 

A control system, given normally by a differential equation 
of the type 

x=f(x,t,u ) ( 1 ) 

where * is an n vector (the state variable), / is a real variable 
(the time), and u is an m vector (the control or steering function), 
can be regarded as a more general system than a simple differen¬ 
tial equation. Indeed, such an equation assigns to each point x 
and time t a definite direction vector; meanwhile the control 
system (1) assigns to the couple *, t a whole set of possible 
values of *, according to the choice of the control parameter u. 
Systems, where at each point a whole cone or a more 
general set of possible directions are given, were considered 
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a long time ago by Zaremba 8 and Marchaud 6 . These authors 
obtained the existence theorems and basic results concerning 
such systems. 

The next step of giving a set of axioms governing the motions 
defined by control systems, in order to develop the theory 
independently from equation (1), was undertaken by Barbashin 1 . 
Among later applications of this same idea, Zubov 9 must be 
mentioned; he gave stability theorems using Liapunov-type 
functions for systems defined axiomatically (using a different set 
of axioms to Barbashin); but apparently these works were more 
influenced by the classical results about dynamical systems, 
trying to include the case of non-unique solutions through each 
point rather than by the problems arising in control theory. In 
this paper it is attempted to show that with this axiomatic 
approach one can obtain results of important significance in 
control theory. 

Notation 

Consider a complete locally compact metric space X = {x} 
of the state variable x. The real variable teR = (— oo, + oo) 
is called time. In order to simplify the ideas only autonomous 
systems are considered, but most results can easily be generalized 
for time-dependent systems. 

Point sets in X are denoted by capital letters A, B, the 
elements of X and also real numbers or functions, by small 
letters x, y, ... The distance defining the metric in X is p (x, y). 
Sometimes, in order to avoid infinite distance between sets, this 
metric can be replaced by p (x, y) //-bp (x, y), but in this paper no 
difficulty will arise in this aspect. The distance between a point x 
and a set A is defined by p (x, A) — p(A,x) — inf. {p (x, a); aeAj. 
‘Separation’ of the set A from the set B is called the number 
P* (A, B) -- sup. {/) (a, B); a e A}. It should be noted that this 
separation is not symmetric in A, B ; in general, that is 
p* (A, B) p* (B , A). The ‘distance’ between sets A and B is 
then p (A, B) max. {p* (A, B)\ p* (f?, A)}. In the space of 
the compact subsets of X , this distance function defines a true 
metric 4 ’ 5 , if one admits closed but not compact subsets, this 
distance can become infinite (and this can be avoided by the 
use of the above-mentioned change of the distance function, 
which does not change the topology). If one also wishes to take 
into account non-closed sets, it defines only a pseudo-metric. As 
compact subsets are dealt with, further details will not be given. 

Finally, S r (A) {x; p (x { A) ^ r) will denote the 
r-neighbouring set of A ; note that this is a subset of X and not a 
neighbourhood of A considered as an element of a space of 
subsets. 

The Attainability Function 

Basic Axioms 

The state-point xe {X} will change with time t, according 
to conditions inherent to the system and to some specific control 
action. Before choosing any particular control function, the 
only thing which can be stated is the whole set of points of X 
which are attainable, in the time t, from a given x. So the system 
is characterized, giving a function F (x, t ), which is a subset of 

with the meaning ye F(x, t) if, and only if, starting at x for 
t = 0, one can reach the point y at the time t by a suitable 
choice of the control function. 


Disregarding formally that meaning of the attainability 
function, it is assumed that the following axioms are fulfilled: 

(I) F(x, t) is defined for every x e X, t ^ 0, and is a closed 
non-void set. 

(II) Initial condition: F(x, 0) = {x} for every x. 

(III) Semigroup property: if 0 t x ^ t 2i then 

F (%o ^ P G ^l) 

xi eF(xo, ft) 

(IV) Given x x f ^ 0, there exists (at least one) x 0 such that 
Xi e F(x x , tj). 

(V) Continuity with respect to /: F(x, t) is continuous in t. 

This means that, given x 0 , t Q ^ 0 and s > 0, there is a d > 0 
such that for any / ^ 0, 1 1 — /-Jed, the inequality 

p [F (x 0 , t 0 ) 9 F(x 0 , t)] < e is satisfied. 

(VI) Upper semicontinuity with respect to x: Given x 0 , 
*o = 0 and s > 0, there is a <5 > 0 such that for any x e X, with 
p (x, x 0 ) < <5, the inequality p* [F(x, t Q ), F (x 0 , / 0 )] < s is 
satisfied. 

By examples it can be shown that these axioms are inde¬ 
pendent of each other. It is interesting to note that axiom (III) 
is equivalent to the following two, which is formulated only in 
words: 

(III cl) If x x is attainable from x 0 in the time t l9 and x 2 is 
attainable from x x in the time r 2 » then x 2 is attainable from x 0 in 
the time t x + t 2 . 

(Ill b) If x 2 is attainable from x 0 in the time t 2 , and 0 ^ t x ^ t 2i 
then there is some point x x attainable from x 0 in time t l9 such 
that from it x 2 is attainable in time t 2 — t v 

Barbashin 1 gives an equivalent set of axioms, and assumes 
the function F(x, t) defined for all real values of t. It seems, 
perhaps, more natural to the essence of control theory to start 
assuming only positive values of t (evolution to the future, 
disregarding the past). Zubov 9 also considers this kind of 
systems for positive times, even without stating explicitly these 
axioms. 

In a straightforward way the set function 
F(A,t)= U F(x,t) 

X £ A 

can be defined; for this function properties similar to axioms (I) 
to (VI) hold. If one only considers compact sets A, the similarity 
is complete, but if allowing all non-void subsets of X, then it 
must be remembered that they constitute only a pseudo-metric 
space, which is not Hausdorff 6 . 

The systems satisfying these axioms include, for example, the 
differential systems which fail to satisfy the condition of unique 
solution (i.e. the Lipschitz condition). If, instead of the upper 
semicontinuity of axiom (VI), regular continuity of F(x,t) 
with respect to x is required, one excludes these systems, but 
still includes the technical important control problems. 

Consequences of the Axioms 

Trajectories —The most remarkable consequences of the 
given axioms follow. 

The set F (x, t) is compact. If A is compact, then so is F (A , t). 
This is a consequence of the continuity axiom (V). It rules out 
the possibility of solutions with a finite escape time. 

One can extend the domain of the attainability function to 
negative values of t, defining: y e F(x, ~~t) if, and only if, 
xeF(y, t) ( t > 0). All but the continuity conditions are then 
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also satisfied for negative values of t. On the other hand, it is 
possible to have ‘finite escape times’ for negative values of /, so 
that F(x, t) is no longer compact. To avoid this, one has to 
assume an additional hypothesis, but it is not necessary for the 
further development of the theory. 

A trajectory is defined as a curve x (t ) in some t interval, 
such that if t x < t 2 belong to that interval, x (t 2 ) eF[x ( 4 ), t 2 ~- /J. 
A fundamental theorem states that if y e F (x, t ), then there 
exists some trajectory going from x to y (supposed t > 0). The 
notation x =<p (x 0 , t) is used here for a trajectory. Barbashin 
has proved the following strong theorem: if x n -> x 0 and 
$n t) is a sequence of trajectories through x n , then there 
exists a subsequence tending (pointwise) to some limit trajectory 
passing through x 0 : <p n (x n , t)~^(j) 0 (x 0 , /). 

Strong and Weak Properties of Control Systems 

Now one is in a position to extend to control systems all the 
properties normally found in dynamical systems. There are two 
very important ways to generalize any property. Suppose that in 
the theory of ordinary dynamical systems a set is called ‘A’ if 
the trajectories starting at points of that set have the property 
y. In the ordinary dynamical systems there is one well-defined 
trajectory through each point, so that this definition is precise. 
In control systems, instead, there is a whole set of trajectories 
through each point, which makes the direct application of the 
same definition ambiguous. It will then be said that the set A is 
‘strongly A 5 if all the trajectories starting at points of A have the 
property y. On the other side it will be said that the set is 
‘weakly A’ if, starting at each point of A , there is some trajectory 
which has the property "p\ To take an example: the set A is 
called strongly positively invariant if every trajectory starting at 
a point of A remains in A for all positive t. It is weakly invariant 
if at each point of A there starts (at least) one trajectory remain¬ 
ing in A for all t > 0. The meaning is obvious: if A is weakly 
invariant, one is able to remain in A by a convenient choice of 
the trajectory; if, instead, A is strongly invariant, one cannot 
leave A for any choice of the trajectory. In a similar way all 
stability definitions of dynamical systems give rise to strong and 
weak definitions in control systems. 

In order to study stability properties Liapunov’s functions 
for control systems are defined, considering upper and lower 
total generalized derivatives for those functions which corre¬ 
spond to the strong and weak properties to be proved. 

Definitions 

All the following definitions refer to a certain general control 
system, defined by the attainability function F (x, t). 

The set A is a strongly positively invariant set if 

x e A => F (x, t) cz A for all / ^ 0. 

The set A is a strongly negatively invariant set if 

x eA => F{x , t) cz A for all t ^ 0. 

The set A is a ‘strongly invariant set’ if x e A => F (x, t) cz A 
for all t. 

The set A is ‘weakly positively invariant’ if for any xe A 
there is a trajectory </> (x, t) contained in A for all t ^ 0. It can 
be seen 1 that this requirement is equivalent to the following: A is 
positively weakly invariant if x e A => F(x, t) n A # (j) for 
t ^ 0 . Similar definitions hold for negatively weakly invariant 
and weakly invariant sets. 


The set A will be called ‘strongly stable’ if for any e > 0 
there is a S > 0 such that x e S d (A) => F (x, jR + ) c S e ( A ), 
where R + = (0, 00 ). Remark: If A is strongly stable then 
closure A is strongly positively invariant. 

The set A will be called ‘weakly stable’ if for any s > 0 there 
is a <5 > 0 such that for any point x e S s (A) there is a trajectory 
(j) (x, t ) satisfying^ (x, R+) cz S £ (A). 

The set A will be called ‘strongly quasi-asymptotically 
stable’ if for some fixed <5 > 0 and any given e > 0 there is a 
T(e) ^0 such that x e S d (A) => Fix, t) cz S £ (A) for all 
t ^ T(e). 

The set A will be called ‘weakly quasi-asymptotically stable’ 
if for some fixed d > 0 and any given e > 0 there is a T («) ^ O 
such that through any point x e S d iA) there is a trajectory 
satisfying (j) (x, t ) c= S e iA) for all t ^ T is), and (j) (x, t) can be 
chosen independently of e. 

The set A will be called ‘strongly asymptotically stable’ if it 
is strongly stable and strongly quasi-asymptotically stable. 

The set A will be called ‘weakly asymptotically stable’ if it is 
weakly stable and weakly quasi-asymptotically stable. 

The set A will be called ‘strongly quasi-asymptotically stable 
in the large’ if for any a > 0 and any e > 0 there is a /? 0%) > O 
and a T ioc, e) ^ 0 such that x e S a iA) => Fix, t) xz S fi iA) for 
all t ^ 0 and Fix, t) cz S e iA) for all t^Tioc, e). 

The set A will be called ‘weakly quasi-asymptotically stable 
in the large’ if for any point x there is a certain trajectory (j) (x, t) 
such that for any oc > 0 and any e > 0 there is a ioc) > 0 and a 
Tioc, e) :> 0 satisfying x e S a iA)=>(j f> (x, t) cz S (j iA) for all 
t ^ 0 and (j) (x, t) cz S e iA) for all t Tioc, e). 

The set A will be called ‘strongly asymptotically stable in 
the large’ if it is strongly stable and strongly quasi-asymptotically 
stable in the large. 

The set A will be called ‘weakly asymptotically stable in the 
large’ if it is weakly stable and weakly quasi-asymptotically 
stable in the large. 

These definitions are the straightforward translations of the 
classical definitions to control theory, as explained before. In a 
similar way, all refinements of the concepts of stability, e.g. for 
non-autonomous systems, can be translated to control theory 
in a strong and a weak form. 

Characterization of Invariance and Stability by Means of 
Liapunov’s Function 

Similar to the case of ordinary dynamical systems, the 
invariance and stability properties of sets can be characterized 
by Liapunov-type functions. Theorems on the strong form of 
stability were already given by Zubov 9 , the parallel development 
of theorems concerning the weak form can be accomplished 
without major difficulties. Sometimes the proofs are more subtle 
because of the fact that if for some sequence 0 < h< t 2 < t z ... 
one has x< e A(x 0 , t z ), it is not possible to deduce from this that 
there exists a trajectory passing through all Xi (this is not in 
general the case). Nevertheless, the following theorems are given 
without proof as examples of the power of the second method 
of Liapunov applied to these problems; before stating them 
some definitions must be given. 
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If A is a given set, it is said that the scalar function V (x) is 
positive definite (A) if: 

(i) V (x) is lower semicontinuous [this is, lim V (y) ^ V (x)]. 

y-*x 

(ii) V(x) ^ 0 for x e A, V (x) > 0 for x A. 

(iii) There exist two continuous monotonically increasing func¬ 
tions v (r) and w (r ) of the real variable r 0, such that 
v (0) = w (0) — 0 and v [p (x, A)] g V (x) ^ w [p (x, A)]. 

From this definition it follows that the set A is closed. 

One calls generalized upper total derivative the expression 


V 


(x)= lim l.u.b. 

t-+ o + 


VjF(x, t )]- 

T 


•V(X\ 


0<T<t 


Similarly the generalized lower total derivative is the ex¬ 
pression 


V (x)= lim g.l.b. 

f-** o + 


J >[F(x,t)]-K(x) . 


; 0 <t <|f 


The generalized total derivatives (x) and y~ (x) are 
defined in the same way with 0~, 0 > r ^ t. The generalized 
total derivatives y(x) and y (x) correspond to t-> 0 , —t r ^ 
Now we are able to state the following theorems. 


Theorem 1: If V (x) is any function such that y + (x) ^ 0, then 
the set A (A) = {x; V (x) ^ X}, supposed non-empty, is a 
strongly positively invariant set. 


Theorem 2: If V (x) is lower semicontinuous and y + (x) sg 0, 
then the set A (A) = {x; V (x) ^ A}, supposed non-empty, is a 
weakly positively invariant set. 


Theorem 3: If A is a closed set, V (x) is positive definite (A), and 
~j/ + (x) 0, then A is strongly stable. 

Theorem 4: If A is a closed set, V (x) is positive definite (A) and 
V + (x) ^ 0, then A is weakly stable. 

Theorem 5: If A is a closed set, V (x) is positive definite (A) and 
~p + (x) is negative definite (A), then A is strongly asymptotically 
stable. 


Theorem 6: If A is a closed set, V (x) is positive definite (A) and 
(x) is negative definite (A) } then A is weakly asymptotically 
stable. 


In all these theorems the function V (x) is supposed to be 
defined in a suitable neighbourhood of the set A. The importance 
of the theorems is due to the physical meaning of the given 
definitions and to the fact that the upper and lower generalized 
total derivatives can be (as a matter of principle) evaluated 
without any reference to trajectories or possible choices of 
control functions. In the definitions of those derivatives, only 
the behaviour of the V (x) functions of the attainable part of 
some neighbourhood of x 0 plays any role. In very complicated 
systems, for example biological or social systems, the lack of 
knowledge of the equations governing the motion, or of the 
kind of control, does not necessarily directly concern this 
approach. 


Relation with Classical Results 

If one wishes to introduce differentiability conditions which 
approach the classical cases, one considers the space X to be 


Euclidean ^-dimensional; to make matters more appealing to the 
geometrical intuition it is supposed that the nth coordinate x n 
represents the time t (as is usually done for passing from non- 
autonomous to autonomous systems). The expression 


C(x 0 )= lim 

f -*0 + 


F(x o , 0 -x o 

t 


has to be understood as a limit of a variable set in the Hausdorff 
metric for subsets of X. If this limit exists and is compact, it 
represents a cone with vertex in x 0 , defining the tangents to all 
possible trajectories. More generally, a trajectory x ( t ) will be 
admissible if for any sequence t Q + (i= 1 , 2 ,3,...) such that 
xi — x (ti) -> x 0 and there exists the lim x r x 0 / t r t 0 , this limit 
belongs to the cone C (x 0 ). But this is exactly the definition of an 
equation in contingents 5 . The definition can be slightly generalized 
in the way that limits of the quotient of the type x; -Xj j t r tj are 
considered and then it is called an equation in paratingents 8 . 
These were considered by Zaremba and Marchaud, as already 
mentioned. Under some assumptions about continuity and 
convexity, a purely geometrical approach to Pontrjagin’s 
maximum principle in the theory of optimal control can be 
made 7 . By linking all these intermediate steps together, one 
can expect to obtain many known results in a purely axiomatic 
way. Of course, this will be of real advantage only if the assump¬ 
tions made are really more general, but if only the results ob¬ 
tained appear more naturally and clearer, this approach will 
also be useful. 

Finally, mention must be made of the interest which might 
result in the study of systems which do not satisfy all axioms. 
For example, if axiom (IV) is omitted, systems are included 
where some states are not the result of the evolution from some 
previous state, but give a starting point for a new evolution. 
The author is not in a position, and it would not be in the scope 
of this paper, to discuss the possibility of application of such 
systems. 
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DISCUSSION 


Author’s Opening Remarks 

The following are three recent publications about, or related to, the 
subject of this paper: 

(1) Roxin: RJ.A.S . Tech. Rep . 62-16. In this work all the proofs 
mentioned in the paper are given for the more general case of non- 
autonomous systems. 

(2) Wazewski: Several papers in the Bull. Acad. Polonaise de 
Sciences , since 1960, developing the theory of contingent equations 
(Marchaud-Zaremba). 

(3) Bushaw: R.I.A.S. Tech. Rep. 63-10. ‘Dynamical polysystems 
and optimization’, where the basic element defined axiomatically is 
the trajectory. 

It would be interesting to establish the conditions under which 
these different approaches are equivalent to one another. 

W. De Backer, Cetis Euratom CCR , Ispra, Italy 

Supposing that a set of differential equations describes a dynamical 
system, then the same can be said of the corresponding attainability 
functions. Between both descriptions there exists a mathematical 


relationship; this relationship, however, can be very complicated as 
far as the computational aspects using electronic computers are 
concerned. This means that it would not be the same for an electronic 
computer if some problems were stated in terms of differential equa¬ 
tions or in terms of attainability functions. In putting forward this 
remark I have in mind a very important class of optimization problems, 
namely those concerning economical systems. I think, although of 
course it is only a matter of intuition, that most of the present economic 
optimizers, who are individuals, governments and planning teams, 
have much better estimates of the attainability functions than of the 
differential equations of the economic dynamical system they are 
faced with. 

E. Roxin, in reply 

Indeed, mathematical generalizations and axiomatic foundations of 
theories are not only made for elegance of presentation, but mainly 
in order to be able to apply them to cases not included in the original 
theory. Economical, social and biological systems, where the differen¬ 
tial equation governing its behaviour is extremely difficult to know and 
even the existence of such an equation is doubtful, might be good 
examples for application of the present theory. 
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Summary 

The inverse problem of integral square estimation may be treated as 
a problem of inversion of the transient performance of linear stationary 
systems. The present paper discusses a method of solving this problem 
by means of spectra of square integral estimation introduced for this 
purpose. The analytical expressions obtained allow one to deter¬ 
mine in a unique manner the transfer functions corresponding to 
any integral square estimation given by means of these spectra. 
They are therefore particularly useful for the synthesis of simple and 
multiple stationary linear systems; at the same time they enable one 
to select in a simple way the values of the parameters to satisfy the 
required static and dynamic characteristics of the systems. 

Sommaire 

Le probleme inverse de festimation de 1’integrate du carre de 
ltecart peut etre considdrd comme un probleme d’inversion du 
regime transitoire des systemes lineaires stationnaires. Dans le 
present travail on propose une mdthode de solution de ce pro¬ 
bleme au moyen de spectres de fintegrate du carre de l’ecart 
introduits spdcialement dans ce but. Les expressions analytiques 
obtenues permettent de determiner d’une fa?on unique les 
transmittances correspondant aux valeurs quelconques, donnees a 
priori, de l’intdgrale du carre de Tecart donndes au moyen de ces 
spectres. Par consequent ces expressions sont particulierement utiles 
pour la synthese des systemes lindaires stationnaires asservis simples 
ou multiples. De meme elles donnent la possibility de choisir d’une 
maniere suffisamment simple les valeurs des parametres satisfaisant 
aux caractdristiques exigdes, statiques et dynamiques, du systeme. 

Zusammenfassung 

Das inverse Problem zur Bestimmung des quadratischen Integral- 
kriteriums laBt sich als umgekehrte Aufgabe zur Giitebestimmung des 
Ubergangsverhaltens in linearen stationaren Systemen betrachten. 
Die Arbeit erlautert die Losungsmethode dieser Aufgabe; sie stiitzt 
sich auf speziell fur diese Zwecke eingefiihrte Spektren des quadrati¬ 
schen Integralkriteriums. Die erhaltenen analytischen Beziehungen 
ermoglichen es, die Obertragungsfunktionen entsprechend den 
Spektren des quadratischen Integralkriteriums eindeutig zu be- 
stimmen. Deshalb eignen sich diese Beziehungen besonders zur 
Synthese von ein- und mehrfachen linearen stationaren Systemen. 
Die Spektren des quadratischen Integralkriteriums erlauben es gleich- 
zeitig in einfacher Weise die Parameterwerte, die die statischen und 
dynamischen Charakteristiken des Regelungssystems erfullen, zu 
bestimmen. 


Introduction 

This paper is a continuation of the author’s work devoted to the 
investigation of automatic control systems by means of deter¬ 
minant indices of stability margin 1 . One of the principal prob¬ 
lems considered in that reference is the inverse stability 
problem of linear systems; that is, the problem of obtaining 


expressions enabling the characteristic equation for prescribed 
values of the indices of stability margin to be established. 

The paper is devoted to the inverse of integral square 
estimation of transient responses. The inverse integral square 
estimation may be considered to constitute the inverse problem 
of quality of a transient process in a linear system. The solution 
of this problem is obtained by introducing the notion of spectrum 
of the integral square estimation. The expressions obtained 
enables the determination, in a unique form, of the transfer 
functions corresponding to a prescribed qualitative evalution 
according to the integral square estimation of transient responses, 
thus being particularly useful for the synthesis of one- and 
multi-loop systems. 

This work was done under the direction of the Academician 
B.N. Petrov to whom the author wishes to express his gratitude 
for many valuable remarks and suggestions. 


1. The Inverse Stability Problem of Linear Systems 

The starting point of the present paper is the inverse stability 
problem of linear systems. The integral square estimation 
expressed in the form proposed by A. Krasovskii and called, in 
what follows, the Krasovskii integral criterion or the Krasovskii 
evaluation 2 ’ 3 , is the assumed estimation of quality. For the sake 
of comparability of results the normalized Krasovskii evalua¬ 
tions are considered. One has 

Ji m) =Jl m> [F(p)-] (1) 

fc oP m + b l^ m ~ 1+ ••• + b m-lP+l (2 ) 

A(p) p" + a 1 p n ~ 1 + ...+a n - 1 p +1 

where F ( p ) is the normalized transfer function and n > m > 0. 

The consideration of normalized Krasovskii evaluation and 
normalized transfer functions F(p) does not affect the generality 
of the assumptions. 

It has been shown 1 ’ 4 » 5 that the Markov stability criterion 
enables a solution of the inverse stability problem of linear 
systems to be obtained. The generalized notion of determinant 
indices of stability margin has also been introduced 1 ’ 5 ; the 
indices will be denoted by SMI (Stability Margin Indices). 

The determination of the values of the coefficients of the 
characteristic equation corresponding to arbitrary values of the 
SMI is obtained according to the developed method 1 by inter¬ 
mediate determination of the Markov parameters. To omit the 
intermediate stage (the determination and calculation of Markov 
parameters), which is specially convenient in the case of syn¬ 
thesis of linear systems based on the qualitative Krasovskii’s 
integral criterion, a new method has been developed for establish- 


645 




W. JAROMINEK 


ing characteristic equations, corresponding to any prescribed 
conditions concerning the AM/ 6 ’ 7 . It presents a new and 
independent solution of the inverse stability problem. 

2. Expansion of the Coefficients of the Characteristic Equation 
in Terms of SMI 

The new solution of the inverse stability problem consists 
in expansion of the coefficients of the characteristic equation 
in terms of determinant indices of stability margin and, in par¬ 
ticular cases, in terms of the Hurwitz or Markov determinants 
or Routh parameters. As an example of the expansion of the 
coefficients of the normalized characteristic equation in terms 
of Hurwitz determinants, mention should be made of Table 2, 
Reference 1. To generalize the results obtained there to the case 
of any degree 6 /z’ write the characteristic equation A n ( p) = 0 in 
the following form 7 : 

A n(p)=p" + a Un p n ~ 1 +a 2t „p n ~ 2 

+ ■■■+a ki „p n ~ k +...+a„'„ (3) 

By considering the sequence of Routh’s matrices correspond¬ 
ing to successive values of the degree n and the equivalent 
sequence of Hurwitz matrices, it can be shown that the coefficients 
a k % n (k = 1,2, of the characteristic polynomial (3) can 

be expressed in a unique form in terms of Hurwitz determinants 7 . 
In particular, the following expansions of the coefficients a lc n 
are obtained in terms of Hurwitz determinants A fc n = A k : 


a i,n — 


Ai 

A 0 


a _A 2 y A £ _ 1 A i+2 


i = 1 


A; A; 


i +1 

Am 


3 ’ n a/AoA A, A ; +1 


( 4 ) 




2 k-ly 2k “ 


. A 2 /C-I 


+ 


A 2 k 


A 2 i- 


-1 A 2 f-i 


A 2 k - 


2 A ik-X i=i &2i-2 A 2 


( 5 ) 


where A k (j ?) is a polynomial of degree k and A ? , = 1 for 
7=0,— 1,— 2,... The recurrence equation (9) holds for 
1 < k < n (Reference 7). In particular, in the case of k — 1,2 
one obtains A 0 (p) = A- x (p) = 1. 

Analogous expressions may be derived for the remaining 
forms of the SMI 1 . 

Equations (4)—(9) enable the inverse stability problem of 
linear systems to be easily solved. The selection of appropriate 
values of the SMI should be done on the basis of a suitable 
qualitative criterion of transient responses. 

3. The Transformed Krasovskii Integral Criterion 

As an estimation of quality of transient responses assume 
the Krasovskii integral criterion J n < m h it has been shown 1 ’ 5 > 8 
that as a result of a suitable transformation the integral square 
estimation J n W can be expressed in a simple manner in terms 
of the indices of stability margin. The transformed evaluation 
takes, when Hurwitz determinants are used, the form 


1(0) _ 1 ( a 2k-l . 1 ^2^2 , A 4 A 4 

Cl 2k Gi AjA 3 A 3 A 5 


+ ...+ 


A 2 (k— 1 A 2 


(k- 1) 


— Il a 2k~l . Aq ^ A 2 (i-x) A 2 (i-i) 
M A A 


l 2k A 1 itl 2 A 2i ~2 


A2/C-3 A 2fe _ 1 

( 10 ) 


7(0) 

J 2k+l 


a 2k | j A 3 A 3 ^ A 5 A 5 

a 2k+l A 0 A 2 A 2 A 4 A 4 A 6 


+ 


2k- 1 A 2 / c - 1 

A 2 & 


A 2 k 


a 


—+E A 

2k+1 i=l a 2i~2 


A 2 i— 1 . A 2t --j 


00 


In order to obtain a complete transformation of the evaluation 
J n (°) make use of the relations (5)-(7) and express the ratios 
a n~- lla n fo the coefficients of the characteristic equation 
A n (p) in terms of Hurwitz determinants. It is found that 


a 2k, 1 — 


2k A 


A 2 k—1 


_ 0 , y A 2 1 

Al i= 1 A 2 i_i 


A 2 i 


a " ,n ~A 

^n- 1 


( 6 ) 


( 7 ) 


Table 1 has been prepared on the basis of eqns (4)-(7). 

In the general case the expansion of the coefficients in terms 
of Hurwitz determinants is expressed by the following algorithm 7 : 


n _ n i A » — 3 ^n 

a k,n — a k,n-l+~ -I- <2 


( 8 ) 


A A Vl k-2,n-2 

a n~2^n-l 

where a i s s 1 in the case of i = 0 and 0 < s < n 
a oi,a= 1 inthecaseof k = 0, — 1, — 2 ,... 
tfi.s = 0 in the case of i < 0 or i > s 

The expressions for the expansion of the coefficients a kn 
can be most easily obtained by means of the recurrence equations 


Mp)=pA k _ t (p)+A -3 A * Ak _ 2 (p) 


Aft- 2&k-l 


( 9 ) 


A '2 ft 


__ v A 2 i-i A 2i - t 
i= 1 A 2 2 A 2 i 

( 12 ) 

_Ap + v A 2 i . A 2 i 

Ai »=i A 2i _ 1 A 2 i +1 

( 13 ) 


Observe that the evaluations (10) and (11) have different 
forms in case of even (n = 2 k) and odd (n — 2k + 1) degrees n 
of the characteristic equation. Substituting in (10) the expression 
(12) and in (11) the expression (13) and performing an appro¬ 
priate change of summation indices one obtains a single general 
expression 


j(°) = JL y ViA-i 

- Z, A A 


( 14 ) 


0 A 

Z i=l a i -2 

where A 0 = A_ x = 1 is arbitrarily assumed. 

The expression (14) is the transformed Krasovskii evaluation 
expressed exclusively in terms of indices of stability margin*. 

Other forms of the transformed Krasovskii evaluation may be 
found in Reference 7. 
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It holds for both even and odd degrees; that is, for any degree n 
of the characteristic equation. 

The above transformation of the Krasovskii evaluation may 
be considered as a transition from one' set of independent 
variables to another. The independent variables of the first set 
are the coefficients of transfer function; those of the other, the 
SMI. Further investigations show that this transformation is of 
essential importance chiefly because the SMI supply much more 
necessary information on the control system than the transfer 
function coefficients. It is also of importance that the new 
expressions of the integral square estimation take a much simpler 
analytical form, which is essential for the synthesis of control 
systems. 

To generalize the results obtained to systems of the non-zero 
class (m ^ 0; n > m > 0) consider some of the relations 
between the Krasovskii determinants and the SMI. 


4. Expansion of the Krasovskii Determinants in Terms of SMI 


In the general case the normalized f integral square estima¬ 
tion takes the form 


where 


1 nt \(n ) 

z 1=0 


— b. 


(15) 


2 b m _ a + 1 f > m _ a _ 1 - f - 2 fc m _ a+2 £> m _ a _ 2 

+ ...+2(-l)"-«M— 2. (16) 

for a = 0,1, 2, ... w and b m = 1; b k = 0 (k < 0; k > m). 

The expression of the normalized evaluation (15) in terms of 
SMI requires, above all, the expansion of the Krasovskii 
determinants in terms of SMI 7 ’ 8 . The elements of 

these determinants are exclusively the coefficients of the charac¬ 
teristic equation, therefore the unique expansion A^ m -JA n 
in terms of SMI may be done on the basis of eqns (4)—(9). As an' 
example a few of the relations obtained are quoted: 


A (n) "A A 
u » _ y A i—l . A i-i 

A„ £ 1 A i _ 2 A ( 


(17) 


A^ i= A-a 
A n Ai — 1 


(18) 


A n _ * 


(19) 


A/n ~ 3 _ _ 4 A„_3 A„_3 

An An —2 A n _ 2 A n _ 1 


( 20 ) 


The expansions of the remaining expressions of the form 



f The normalized evaluation y n (m ' corresponds to the normalized 
transfer function (F)p > for which one has a n ^= b m ~ 1. 


can be represented in a similar manner. Consider the sequence 



of these expressions: 


Am } , Aw-l . Am-2 . 
A„’ A„ ’ A„ ’ 



( 21 ) 


It can be shown that the structure of the equation obtained for 
the expansion of each particular expression 



in terms of the SMI is independent of the degrees n and m of 
the transfer function polynomials and depends only on the 
ordinal number a in the sequence 



This property is very useful for the generalization of the con¬ 
sidered problem for the case of m> 0. 


5. The Optimum Integral Estimation J n im) in the Sense of SMI 

The value of the evaluation (15) depends on the distribution 
of poles and zeros of the transfer function (2). Assume that in 
the general case the distribution of the zeros is independent of 
that of the poles. Then, the coefficients are also inde¬ 

pendent of the coefficients of the characteristic equation and 
cannot, in general, be expressed in terms of SMI. For any 
assigned distribution of transfer function poles there exists only 
one distribution of zeros of the polynomial B(p) in the numer¬ 
ator of the transfer function, which, for the given assumptions, 
corresponds to the minimum value of the evaluation J n < OT >" 
Such a distribution of zeros will be called, in what follows,' 
optimum in relation to the SMI. The determination of the cor¬ 
responding optimum polynomial S(p)=B(p) opt will be 
called the optimization of the integral square estimation JJ m '> 
in the sense of SMI 7 . 

The determination of the values of the coefficients of the 
optimum polynomial B (p) opt reduces to that of the extremum 
(minimum) value of a function of many independent variables. 
To do this one must equal to zero the partial derivatives of the 
evaluation JJ m ) with respect to the coefficients of the poly¬ 
nomial B (p). One has 


SJM 0JO") 

06 o ’ db 1 


0 ; 



5J-"° 

S&m-l 


=0 ( 22 ) 


Solving successively for each m the system of m equations 
0 (k = 0,1, 2, .„, m — 1) the values of the 
coefficients of the polynomial B(p) opt will be obtained, i.e., the 
optimum in the sense of SMI. Thus, for instance, in the case of 
in — 3 one has: 


b 0 = 


A n -3 

\,-4 


. u _ A« - 2.7 A„ 

= £>,=- 


A„_, 


A; 

■2 A„_ 4 A„_ 


-3 . 


(23) 


Table 2 contains expressions for the coefficients of optimum 
polynomials B O) opt , obtained as a result of solution of the set 
of eqns (22) for a few successive values of the degree m of the 
polynomial B (p). 
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Table 2. Coefficients b t (i = 0, 1, 2, 


.., m) of the polynomials B(p) 

stability margin 1 q t 


B(p) opt, satisfying the optimum conditions in the sense of the indices of 
(• A * 


v ?:+i 

A, 


m 

Coefficients of the polynomial B (p) — b 0 p m + b 1 p m 1 -j- ... + b m ^p + b m — B(p) ov t 

bo 

b i 

b, , ' 

bz 

K 


1 

Qn—2 

1 





2 

Qn- 3 

Qn— 2 

1 




3 

Qn~ 4 

Qn -3 

■ Qn —4 

Qn-2 + ~Z 

*»—3 

1 



4 

Qn- 5 

Qn —4 

, Qn —5 , Qn -5 

?»-3 + - + „ ' Vn-2 

Hn— 3 'in- 4 

, Qn —4 
<7n—2 + _ 

c in-z 

1 


5 

Qn~ 5 

Qn-5 

, 4n—6 , 4n~ 6 i 

, 5 , 

, Qn-4 , 

1 

<ln-i "T ' _. ".b «?n-2 d Hn -3 

Qn —3 $n- 4 'in— 5 

-r + - * tfn-2 

( in—i 

2 Qn- 3 ^n-B 


Notes: (1) n is the degree of the characteristic equation A (p) = 0 

(2) m is the degree of the polynomial B(p); 0 ^ m < n 

(3) B (p) is the polynomial in the numerator of the normalized 

A (p) 

transfer function F(p)= ^ 


The integral evaluation that satisfies the set of con¬ 

ditions (22), will be called optimum in the sense of stability 
margin and denoted by Optimum evaluations in the 

sense of SMI, have a number of valuable properties. Some of 
them will be considered below. Of particular importance is the 
fact that for full analytical description of the evaluation / n (w) op t 
only the SMI are required. 


6. The Two Equivalent Forms of the Integral Evaluation Jj m ~> 

In the general case the integral evaluation J n does not 
satisfy the optimum conditions (22), and therefore it cannot be 
expressed in terms of the SMI only. This follows directly from 
the assumption, that the coefficients of the polynomial B (p) are 
independent of the coefficients of the characteristic equation 
A(p). In this connection try to separate in the integral evalua¬ 
tion a component depending exclusively on the SMI from 
another component in which the influence of the polynomial 
B(p) is taken into account. The introduction of the SMI and 
the notion of optimum conditions in the sense of SM/_enables 
two new equivalent forms of the integral evaluation J n to 
be established, that is: 



(24) 


(25) 


A detailed analysis of expressions (24) and (25) will be shown 
later. Now one is satisfied with the statement that for the 
determination of the first components, that is J n {0) and «/ w (m) o P t> 


only SMI are needed. To find the remaining components, that 
is M n W and A Mj m \ the knowledge of the polynomial S (p) 
is also needed. In particular, the component M n expresses 
the increase of the evaluation J n ^ due to the fact that the 
polynomial increase B(p) = B(p) — b m has been taken into 
consideration, and the component AA/ W ( m ) is the increase due 
to the introduction of the polynomial A B (p) = B(p) — B (p) op t 
in the numerator of the transfer function F(p). For further 
investigation form (25) will be of particular use. 


7. The Primary Spectrum of the Integral Evaluation J W < TO > 


Under the term of primary spectrum of the integral evalua¬ 
tion J^ m) one will understand the expression 

R n = R n (r 1 ,r 2 ,r 3 ,...,r n ) (26) 

The elements of the spectrum R n are r lt r. 3> r n . They are 
related to the SMI by the formulae 


r,-= 


A-i' A-i 
Si-2 'A 


sT-iSt-i . 

Sf- 2 St ; 


(i = 1 >2, n) 


(27) 


where q t are the Routh parameters, A* the Hurwitz determinants, 
and S { * the Markov determinants (A; = a( l ■ S t *). 

Example: 

A 0 _A 1 A 1 _A 2 A 2 _ _ A n _j • A„_j^ 

Vi -~sf ,r2 ~~KK'2 r3 ~s 1 A 3 , '" , n ~ A„_ 2 -A„ 

Knowing the values of the elements r„ r 2 , ..., r n the values of 
the corresponding indices of stability margin can easily be 
determined: 
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Routh parameters q s 


1 k 

a - =r i r 2 >-,r k = Ur i (fc== 1 , 2 ,.... n) 

^k -l i= 1 

Hurwitz determinants A 7c 


_L 

A, 


k k~ 


1 r k~2 
r 3 5 


r t=n 

n- 1 


k + 1 - a 
'a 


for k — 1 , 2 ,..., n 


(28) 


(29) 


Markov determinants S k *; (£)* = 1 ) 


St 


= r 


,k — 1 k~ 2 


< r k- n r i 

a = 2 


fc +1 - 


1 for k = 2, 3 ,n 


(30) 


The primary spectrum R n determines uniquely the first 
components of the forms (24) and (25) of the integral evaluation 
Jn m) - In particular, by virtue of eqns (14) and (27), one can 
write at once 


J n 0) ~Y^ r l +r 2+ ■■■ +0 = 4~ Z r i 


i=l 


(31) 


of this system is essentially positive and bounded, that is 
0 < r t < oo for i = 1 , 2 , n. 

Property II: The primary spectrum R n characterizes the 
transient performance in a linear control system of the order n 
and class m, because the sum of its weight elements r { (i — 1, 
2. ...,n - m) determines the value of the evaluation 
satisfying the optimum conditions in the sense of stability 
margin (SMI), that is 

i n-m 

E r, 

z > = i 

for m = 0 , 1 , 2 , 1 . 


8. The Secondary Spectrum of the Integral Evaluation Jj m) 

The components and AM„< m > in expressions (24) and 

(25) for the evaluations depend in the general case on the 
spectrum R n and the polynomial B(p) = b 0 p m + b x p m_1 + ... 

+ Kn-i P + b m or the equivalent polynomial C(p)=E(p), 
where 


It can also be shown 7 that when the optimum conditions (22) 
are satisfied, the expression of the evaluation J w < m > opt takes 
the following exceptionally simple form 

•C ) P , = yO - 1 + r 2 + ... +r n _ J=— ^ (32) 

^ i = 1 

where 0 < m < n. 

From (32) it follows that evaluation J n ^ m ^ op t depends only on 
the first n-m elements of the spectrum R n and is invariant in 
relation to the remaining ones. Thus, the elements r l5 r 2 , .. 
r n-m will be called weight (influence) elements and the remaining 
ones, that is r n -^ m ^ r n -( m - 2) , ..r ni independent or free ones*. 
Observe that although the independent elements of the spectrum 
R n show no influence on the value of the evaluation J n (m) opti 
they influence the character of the transient response. This is a 
separate problem and is not dealt with in this paper. 

On the basis of eqns (27)-(30) the stability of a control 
system can easily be analysed. From this analysis it follows 
that if a control system is stable, all the elements of the spectrum 
R n are positive. If, in addition, the system is physically real, 
these elements are bounded. The spectrum R n , of which all the 
elements are different from zero and positive will be called 
‘essentially positive’. 

Another interesting property of the spectrum R n is now 
shown. It is known that the stability margin of the system is 
greater for greater values of SMI 3 * 4 , Hurwitz determinants, 
for instance. This means that the stability margin is greater for 
smaller values of elements of the spectrum R n , 

On the basis of the above results and considerations the 
following cardinal properties of the spectrum R n can be for¬ 
mulated: 

Property I: In order that a linear control system with the 
characteristic equation A(p) = p n + a lP n ^ + ... + a n = 0 is 
stable and physically real it is necessary and sufficient that the 
primary spectrum R n - R n (r 1? r 2 ,..., r n ) of the evaluation 

* In the case of normalized transfer function the condition 
n^l should be satisfied. 


C(p) = c m p m + c m „ 1 p m ~ 1 + ... + c 1 p + c 0 =B(p);(c 0 = b m =l ) 

(33) 

The task now is to find a set of m parameters such that their 
structure contains as much information as possible on the 
transient performance in a control system and would enable 
the determination, in a unique form, of the values of the coeffi¬ 
cients c a == b m — a and the easy computation of the component 
M n { or A M n ( m ) of JJ m \ To this aim consider the partial 
derivatives 


for i = l, 2 . 


8c i a = i 0C, 

One has, in the case of odd i: 

E j~ m + 1 j 

£ 

= i 

£ c-^j 

W 2i+1 = Z (-l) (l + ,+ 1) -c at _ - *s=S!+!l 
1 = 1 A n 


, m 


w ; =-l+ X (-l) i+ 1 -c 2i _ 1 


A (n) • 

. “m — i 


and 


where 1 < l < E [m —1/2]. One finds, for even i: 


(34) 


(35) 


E r un 

, A( n ) L 2 J A („) 

w 2 l =(-iy-^+ z (-i) (i+i) - C2; ^ 

i — 1 A n 


+ 0 


(36) 


for 1 ^ / < k = E [m/2]. 

The expressions (35) constitute a set of equations for the odd 
coefficients c a of the polynomial (33) and the expressions (36) 
are a set of equations for even coefficients c a of that polynomial. 
The principal determinants of these systems will be denoted by 
and W m V) respectively, where 




3w! 0WJ 
3 c t 3c 3 
3w 3 3w 3 
3c x 3 c 3 


W 11 ^13 w 15 ... 
W 31 w 33 w 35 ... 

^51 w 53 w 55 ••• 


(37) 
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wl 2) = 


0W 2 0W 2 

dc 2 0c 4 . 

0W 4 0W 4 

0C 2 0C 4 


w 22 W 24 w 26 . 
w 42 w 44 w 46 . 

W 62 W 64 W 66 


(38) 


or 


w! 1) = 


9(w x , w 3 , w 2 fe _ t , ■■■) _ 

0(Ci,C 3 , 


1 


>t‘ 2 /c — l,***) 2^n-3s 

L + m 


> (?n-lt 


k = 2E 


l + m 
2 


1 


ir Ql_ 9(w 2 ,W 4 ,...,W 2 t , ■■■) __ 

m 9(c 2 ,C 4 , ...,C 2k , •••) < 7 „- 2 <?»- 3 > 




(39) 


A (n) 

— a 


with appropriate signs and 


„ ,,, enrrtrmn if,,,.’ In order that the Krasovskii 

Property of i (w) s hould satisfy the optimum conditions 
integral evaluation « gin (SMI) it is necessary and sulli- 

in the sense of stability mciigi' v . =ft 

cient, that its secondary spectrum is zuo, that is, ", - 0 

pmpoi'thTifflcients of the polynomial O) should l.rst be 
represented in the form 

1,2. m) (43) 


Ci — Ci 0 pt 


-hi a- 


where c iopt satisfy the optimum conditions in the sense of SMI 
and expand M w <“> in Taylor’s series lor (unctions ol more than 
one independent variable 


M[ m \c lopl +h 1 ,c 2opl + h 2 .e 


Ml Opt 


+ h m ) 


dMi; n) , 

= M ( „1, + —jj- + 


1 n opt 


From eqns (37)—(39) it follows that the determinants W m (1) 
and WJ 2) are the Jacobians of the transformation. The elements 
Wij of these Jacobians are Krasovskii determinants 


d 2 M c ;l ) d k - , M‘" ) 

“2T" + - + Tk-l~)T k (44) 

In the general case the derivatives d"A-/ M < M > and the rest R, 
of the expansion (44) are 


and 




Ut m 


R* 


dc 2 

d k M { ' n) 


(45) 

(46) 


_dw 1= Q 2 M ( „ m> 
Wij ~dcj ocfiCj 


(i, j = 1,2,3,. ..,m) 


(40) 


Assume that the system is stable and its spectrum JR n is in- 
variable (constant); then, assume also that the Jacobians W m 
and W m l2> have, in agreement with (39), constant values different 
from zero and positive. From the analysis it follows 11 that in 
this case all the necessary and sufficient conditions are satisfied 
for the transformation considered to be homeomorphic. It 
follows that the transformation of the set of parameters w* (z = 1 , 
2,..., m) in an m-dimensional space L ini ) into the set of para¬ 
meters ci(i = 1 , 2 .m) in an m-dimensional space D (m> is 

one-to-one, and that the homeomorphic representation of a 
space region is a space region and the representation of an arc 
is an arc. The set of the parameters w* (z = 1,2,..., m) will be 
called the secondary spectrum of the integral evaluation J n (m) 
and denoted by 

W m = w m (w v w 2 , ...,w m ) (41) 


If the values of the elements of the spectrum w m are known, 
it is easy to calculate all the coefficients c ? of the polynomial 
(33). To do this it suffices to solve in relation to c x the matrix 
equations: 

ll^ 1) ll’l|CL 1) ll = lil / m 1) ll and ||^ 2) II’IICL 2) II = II^ 2) I1 (42) 

The spectrum w m is called positive, zero or negative if all 
its elements w { (z = 1 , 2 , ...,m) are, respectively, positive or 
zero or negative. A spectrum w m may also be of a mixed type. 
In particular, from the solution of eqns (42) it follows that if 
the spectrum w m is zero, the set of eqns (22) is satisfied. This 
important feature of the spectrum w m concerning the optimum 
evaluation J n im) in the sense of SMI can be expressed in the 
form of the following. 


where the derivatives d"M n <"‘> for z-< l< should be determined at 

the point Qo P t= Q opt (ffi opt, o> op ,.c„, op t) and the rest R k at 

an intermediate point (C 10 pt 1 6)/:,, c 2(lp t I tvh. it ..., t'mopi 
&h m ), where 0 < 0 < 1. One obtains 


d M< m) = 


y ’ n h r - 

0 <) ' 


I "’A- 

i ■ 1 


(47) 


^ |<N 

il 

c4 

II 

* 

f m 

T, 

dM <m) \ 

-l: *■) 

2 t m 

-; i»«»?+ 

~ i \ * 

where 






Wij 

dcficj 


m 



i*./ : 
(/<./> 


w u hihj 

(48) 


The expansion of the function M n M in Taylor’s series, taking 
into account eqns (47) and (48), will now be written 


or (49) 

AM< m) = M ( „ m) - + R 2 

In agreement with the optimum theorem of the evaluation 

J„(“) the point Q opt (c x opt , c 2opt .<',„„„,) corresponds to a zero 

spectrum w m . In other words the derivative (47) is at this point 
equal to zero, <lAC"(e,>=<> < 5l » 

It can easily be shown that the second partial derivatives w rj 
do not depend on the choice of the intermediate point. There¬ 
fore the expression of the component of the evaluation 

J n ( m) takes the following very simple form 


A M< m) = R 2 


1 

2 


2 >„/,, 2 + £ 


tJ 
< Jy 


Wijhij 


(w,-,i+i=0) (51) 
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The partial derivatives w u and w tj are Krasovskii determi¬ 
nants taken with an appropriate sign, therefore they depend on 
the spectrum R n only. Analogous considerations show that the 
transformation of the set of parameters into the set of para¬ 
meters hi (i = 1 , 2, m) is also homeomorphic; that is, one- 
to-one. In this connection the parameters will be taken as 
elements of the second, equivalent form of the secondary spec¬ 
trum w m ; that is, 

(/*i, h 2 ,..., h m ) = w w (w x , w 2 ,..., w m ) (52) 

If the spectrum w m is zero; that is, hi = 0 (/ = 1 , 2, ..., m)- 
then A = 0 and J n ( m > = / n W opt . 

The secondary spectrum w m has a number of properties 
facilitating the qualitative analysis of the influence of distribu¬ 
tion of zeros of the transfer function on the transient per¬ 
formance 7 * 12 . Thus, for instance, a positive or negative spec¬ 
trum w m shows that the corresponding fluctuations of transient 
responses are greater or less than the same fluctuations for the 
evaluation / w (m > op t- 

9. The Inversion of the Integral Evaluation JJ m ) by means of 
the Spectra R n and w m 


transfer function F(p) = C(p)/A(p) of the control channel 
under consideration and much information on the transient 
performance in this control channel. 

The transfer function will be determined by inversion of the 
evaluation expressed in terms of the spectra R n and w m ; 
that is, by inverse transformation of the spectra R n and w m . 
For this purpose determine first the characteristic equation 
corresponding to the spectrum R n . 

The method of determining the characteristic equation 
(polynomial) A(p) = p n + a lP n ~' + ... + a n ~iP + ^ is an 
elementary one. The values of all the elements r l9 r 2 , r n of 
the spectrum R n being known, it is easy, on the basis of (29), 
to determine, for instance, the values of all the Hurwitz determi¬ 
nants and then to make use of Table 1 which enables the values 
of the coefficients of the characteristic polynomial A(p) to be 
found directly. 

In order to avoid the intermediate stage of computing the 
Hurwitz determinants. Table 3 has been prepared, containing 
expansions of the coefficients of the characteristic equation 
directly in terms of the elements r< (/= 1 , 2 ,...,«) of the 
primary spectrum R n . In this case the coefficients a kv may be 
determined by means of the algorithm 


To estimate the transient performance in a control system 
various integral criteria have found broad application. This is 
done most often by a comparative method. The less is the value 
of the integral evaluation chosen, the higher is the quality of the 
transient response. In this connection various methods have been 
developed for investigation of the relation between a change of 
values of selected transfer function parameters and the corre¬ 
sponding change of the value of the integral evaluation. Of the 
best known and most widely used, mention should be made of 
methods of minimizing the integral evaluation in relation to one 
or a few parameters: graphoanalytic methods of determining 
the minimum evaluation and the method of successive trials 
and approximations. 

The aim is to obtain analytically a new solution of this prob¬ 
lem using the integral square criterion of transient performance 
which was_called the inverse problem of the integral square 
estimation J n ( w ) and which could also be called the inverse prob¬ 
lem of transient performance. This is a problem encountered 
particularly in the synthesis of linear systems. 

_ Under the name of inversion of the integral square estimation 
J n (m) one will understand the determination of the normalized 
transfer function for a prescribed value of the normalized 
Krasovskii integral evaluation / n ( m \ The solution of the inversion 
problem of the integral criterion has been obtained by 
introducing the notions of the spectra R n and w m defined above. 
It should be explained that in the general case the evaluation 
J n {m) is a multivalued function. For any assigned value of the 
evaluation J n < m ) an infinite number of various linear transfer 
functions can be made to correspond. A different case is that 
where the evaluation J is expressed in terms of the spectra R n 
and w m , the correspondence between a transfer function and 
these spectra being now one-to-one. In this connection, if the 
inverse evaluation/^™) is spoken of, one always means the in¬ 
verse evaluation J n ( m ) expressed in terms of definite spectra 
R n and w m . 

m Assume that the spectra of the Krasovskii evaluation J n ( w ), 
primary R n = R n (r l9 r 2 , ..., r n ) and secondary w m = w m 
(/*!, h 2 ,..., h m \ are known. They contain full information on the 


^k, n &k, n-l~b 


l k- 2, >i — 2 

r n -i'r n 

or by means of the recurrence equations 


(53) 


A(P) = PA _ t (p) + — 2 ^ (fc= 1,2, n) (54) 


r k-i-r k 


The expressions (53) and (54) are equivalent to eqns ( 8 ) and (9) 7 . 
In the case of low degrees n in the characteristic polynomial it 
is more convenient to use Table 3. In the case of high degrees n 
eqns (53) or (54) are better suited for computation. 

The method for finding the polynomial c(p) in the numerator 
of the transfer function F(p) = C(p)/A(p) is also elementary; it 
consists in representing the polynomial C(p) as a sum of two 
polynomials 


C (p)~c m p m + c m _ l p m x + ... +c 1 p + l = C(p) opt 4 -AC (p) 


where 


(55) 


C(p )opt C moptP + C m-loptP” 1 + ... + opt P + 1 | 

&C(p-)=h m p m + h m _ 1 p"'-i+ ... +hlP j W 

The coefficients c a0pt (a = 1,2, m) of the polynomial C{p) opt 
can easily be obtained from Table 2 and the spectrum R n . One 
has only to take into consideration the relation c a0pt = b m - xopt 
between the coefficients of the equivalent polynomial C(p) opt 
and S(p) 0 pt- The coefficients (i =1,2,..., m) are known if 
the spectrum w m is known. As a consequence the transfer func¬ 
tion F(p) = C(p)/A(p) is uniquely determined. The spectra R„ 
and w m enable one to estimate simultaneously the stability of 
a system and the transient performance in agreement with the 
principles studied in Sections 6 - 8 . 

The method based on the inverse integral criterion JJ m ~> and 
the spectra of the evaluation may be used successfully for 
analysis and synthesis of linear systems of automatic control. 
In the first case the transfer function is known, therefore the 


652 



THE INVERSE PROBLEM OF INTEGRAL SQUARE ESTIMATION OF TRANSIENT RESPONSES 





653 


w. JAROMINEK 


corresponding spectra of the evaluation can be found 
easily; in particular, the primary spectrum R n can rapidly 
be determined by using, for instance, the Markov criterion or 
finding, by successive elimination, the elements r i (i = 1 , 2 , 
directly from the expansion of the characteristic equation 
in terms of these elements. The next stage is that of correction 
of the spectra obtained. 

In the case of synthesis it is necessary to know the general 
structure of the transfer function (that is n and m must be 
known) _and the requirements concerning the evaluation 
J n ( m ) — J n ( m ) opt -f ACorrect choice of the weight 
elements r { (/ = 1 , 2 ,..., n — m) and the elements h { of the 
spectrum w m is of particular importance. It follows that the 
problem of correct choice of the spectra R n and w m is essential 
for the synthesis of linear systems and the necessary correction 
of such system. 

10. Minimization of the Integral Evaluation J n ( m ) 

Some additional data on the primary spectrum R n may be 
obtained by minimizing the integral evaluation J n ( m f The 
minimization of the evaluation J n ( m ) = J n < w > opt + AMj m >, 
expressed in terms of the SMI may be divided into two stages! 
The first consists in obtaining the optimum integral estimation 
in the sense of SMI, discussed in Section 5. As result the 
component A M n ( m ) vanishes. 

The next stage consists in minimizing the component J w < w ) 0 t 
in relation to a selected group of SMI, for instance 


11. The Problem of Correct Distribution of Zeros of the Transfer 
Function 

The secondary spectrum w m characterizes indirectly the 
distribution of zeros of the transfer function, which has a strong 
influence on the value of the evaluation JJ m K As a criterion 
of correct distribution of the zeros of the transfer function in 
relation to the distribution of the poles, the value of the com¬ 
ponent AM w ( m > > Oof the evaluation J n < m ) = 7 n < m ) opt -f AM/ m ) 
is taken. The component AAf n < w > depends on both spectra R n 
and w m . The spectrum w m , which is non-zero in general, will 
be correctly chosen to fit the spectrum R n if the component 
A M n ( m ) does not exceed a certain fixed value. 

The upper bound of the value of the component A 
can be determined assuming as a principle the maximum 
utilization of the polynomial C(p) of degree m. For this will 
be used the first equivalent form of the integral criterion JJ m) , 
that is 

and + (61) 

On the basis of appropriate considerations the following can 
be written: 

lim inf. M< m) = M ( n m J pt < M<"° < lim sup M ^~ 1 > (62) 

It follows that 



On the other hand it is easy to show that 


H=H(A U A 2 ,A„) 

R=R (<lo,<h,-,q„-i) ( 57 ) 

M = M(q 0 ,q 1 ,...,q n _ 1 ) 

where H, R, M are groups of determinant indices of stability 
margin in the sense of the criterion of Hurwitz, Routh and 
Markov, respectively. Of course, the result of the minimization 
process depends on the choice of the group of independent 
variables H, R or M. 

As a result of investigations a new property of the spectrum 
has been found, characteristic of the conditional minimum of 
the evaluation This is as follows: 

Property III: An integral evaluation J TC < TO > minimized with 
respect to an appropriate group of SMI reaches a conditional 
minimum if its secondary spectrum w m is zero and the weight 
elements of the primary spectrum R n constitute an arithmetic 
progression of which the difference is A r { and Ar a - = ic/(n— m) Aj; 
r i+i = + Ar s (/ = 1, 2 ,.n — m; m > 0). 

The conditional minimum of the evaluation J n ^ (m > 0) 
is expressed by the equation (Jarominek 7 ), 


4A 1 

In the particular case of 


(58) 


k= -1 one has (H) min =JJ»> ( K =-1 ) min =- ^ + 1 ( 59 ) 

and 4Al 

k = 0 one has JJ"> (K) min = ( K = 0 ) min = (60 ) 


(64) 

It is inferred that AM n ( m ) should be contained between the 
limits 

0<AM^<lr„_ m+1 (65) 

The secondary spectrum w m is therefore correctly chosen if 
condition (65) is satisfied. 

The above method, based on the solution of the inverse 
stability problem and the inverse transformation of the integral 
square estimation J n ( m ) may also be successfully used for the 
analysis and synthesis of multi-loop linear systems of automatic 
control. In the case of synthesis it enables the parameters 
iollowmg the requirements concerning the static and dynamic 
characteristics of the system to be chosen 7 . It may also be helpful 
for the investigation of non-linear systems in the applicability 
limits of the Liapunov theorems 10 , and for the investigation 
of some adaptive systems. 
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DISCUSSION 


P. C. Parks, Department of Aeronautics, The University, Southampton, 
England 

There are some interesting connections between the work of Jarominek 
in this paper and in his earlier work (Reference 1 of the paper) with 
Liapunov’s second method, and work by Schwarz 1 in Switzerland and 
Kalman and Bertram 2 in the U.S.A. If one considers the possibility 
of writing eqn (3) in the special state space form 

x — Bx 

where B is the Schwarz matrix 

B= 0 1 0 0... 0 0 0 

-b n 0 1 0... 0 0 0 

0 -b n „ x 0 1... 0 0 0 

0 0 0 O...-63 0 1 

0 0 0 0... 0 -62 -t>i 


and 

j< 0) =|”x„ 2d t=U ? {^ + b 1 fc 2 ^+b 1 M 3 ^+..j 
= 1{J_ + A + _aL + _aL + ... + _A^1 

2 (A^ A 2 A^Aj A 2 A 4 A n _ 2 A n j 

which is Jarominek’s expression (14). This is minimized for the 
normalized system 



when A« = + 1 (/ = 1, 2,..., n— 1) as mentioned by Jarominek in 
his earlier paper. This leads to ‘optimum’ transfer functions 1 and 
corrections to the analogue computer work of Graham and Lathrop 5 . 
There are further interesting ramifications of the Liapunov function V 
when a linear transformation to the phase space is carried out 6 ’ 7 . 


then Parks 3 has shown that 


b 1 — A1, 



b* = 


A x A 2 


b r = 


A r - 3 A r 

Af.-2Aj.-1 


(r>4) 


where the A r are the Hurwitz determinants of (3). For the state space 
equation above, Kalman and Bertram 2 found the Liapunov function 

V=b 1 b 2 ...b n x 2 1 + b l b 2 ...b n - 1 xl + ... + b 1 xZ 

for which —2 b t 2 x n * which is negative-semidefinite and not 
identically zero along any trajectory, except the origin itself. Necessary 
and sufficient stability conditions follow from the positiveness of the bi 
and hence, from the work of Parks 3 , the A i. Moreover, the expansion 
of det (XI — B) from the bottom right-hand corner leads to a solution 
of the ‘inverse stability problem’, using the expressions for the bt in 
terms of the A$. 

Further, if one considers 

x = Bx + df(t) 

where d = (0 0 0 .... 0 1) and f(t) is a unit step function then 


J<°> |U„ 2 d f =A ? {F(0)-F(cx))}=A2F(0) as F(oo) = 0 

for a stable system. The state space motion decays to the origin from 
the starting point 


X„=L X„-l = 


&1 J 

- - , x n _ 2 1 ? A /t - 3 

O 2 


= • 


b 1 

b 2 by 


1 __etc. 


x n -4 — b 3 b 5 5 Xn ~ 5 
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W. Jarominek, in reply 

The results presented by Mr. Parks are very interesting. Especially 
valuable is the indication that there exists a connection between some 
of my results and works of Schwarz (Switzerland) and Kalman and 
Bertram (U.S.A.). Mr. Parks has shown that using Liapunov’s second 
method it is possible, on the grounds of works of the above-mentioned 
authors, to confirm the correctness of some of the expressions obtained 
in a different way in my earlier works. 

From the methodological point of view, I consider as most valuable 
the application by Mr. Parks of the second method of Liapunov to 
solutions of certain problems dealt with in my works, which opens 
new and interesting prospects of further development of the con¬ 
sidered theory of automatic control. 
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Summary 

The definition, the scheme and five basic operations of systems with 
automatic control of configuration are given. For the description of 
a complex with many elements, division into zones and statistical 
characteristics as state variables are introduced. The configurational 
redundancy and simpler measurable quantities are order measures. 
The probabilities involved in the configuration are influenced 
by the formator through controlled probabilistic transducers. The 
methodical approach to the solution of the formator and complex in¬ 
teraction consists in the formation of a mathematical model, checked 
against physical measurements and in using it in the choice of the 
control algorithm. The two basic deviations and cubic matrices of 
variables of this type of multivariable systems are introduced. The 
optimization of the process of configuration is described using the 
method of Howard in terms of a Markov process with decisions and 
rewards. Three examples of systems are suggested, pertaining to 
formation of strips of elements and to the migration of elements in 
one-dimensional and two-dimensional arrays of zones. Further 
progress is expected from physical modelling of non-homogeneous 
Markov processes. The field of application of the theory can be seen 
in chemistry, in automatic cultivation of algae and in random net 
formation. 

Sommaire 

On donne la definition, le schema et cinq operations de base des 
systemes avec la commande automatique de la configuration. Pour la 
description du complexe avec un grand nombre d’elements on in- 
troduit sa division en zones et des caractdristiques statistiques comme 
variables decrivant son etat. La redondance de configuration et des 
quantites mesurables plus simples sont des mesures de l’ordre. Les pro¬ 
bability concernant la configuration sont influences par le formateur 
par Pintermediaire de convertisseur commandes de probability. L’acces 
methodique a la solution de Pinteraction du formateur et du complexe 
reside dans la formation d’un modele mathematique qui est k verifier 
par des mesures physiques, et dans son emploi pour le choix de 
Palgorithme de commande. On introduit les deux ecarts de base et les 
matrices cubiques des variables de ce type de systemes a variables 
multiples. On decrit Poptimisation du processus de configuration, 
en employant la methode de Howard, en tant que processus de 
Markov avec des decisions et des recompenses. On propose trois 
exemples, concernant la formation de chainettes d’elements et la 
migration des elements dans des dispositifs unidimensionnels et 
bidimensionnels de zones. On attend un nouveau progres de la crea¬ 
tion de modeles physiques de processus non-homogenes de Markov. Le 
champ d’application de la th£orie peut etre entrevu en chimie, pour la 
culture automatique des algues et pour la formation de reseaux 
alSatoires. 

Zusammenfassung 

Die Definition, das Schema und fiinf Grundarbeitsaufgaben von 
Systemen mit selbsttatiger Regelung der Konfiguration werden 
gegeben. Fiir die Beschreibung des Komplexes mit vielen Elementen 
wird seine Einteilung in Zonen und statistische Charakteristiken als 
Zustandsgroften eingefuhrt. Die Konfigurationsredundanz und ein- 
fachere MeBgroBen sind MaBe der Ordnung. Die mit der Konfigur- 
tion verknixpften Wahrscheinlichkeiten werden vom Formator durch 
gesteuerte Wahrscheinlichkeitswandler beeinfluBt. Der methodische 


Zutritt zur Losung der gegenseitigen Tatigkeit des Formators und des 
Komplexes liegt in der Bildung eines mathematischen Modells, das 
durch physikalische Messungen zu priifen ist, und in seiner Anwendung 
zur Wahl des Algorithms fur die Regelung. Zwei Grundabweichungen 
und kubische Matrizen von Variabeln von diesem Typus von Systemen 
mit vielen VeranderungsgroBen werden eingefuhrt. Die Optimierung 
des Konfigurationsprozesses wird beschrieben, mit Anwendung der 
Methode von Howard, als eines Prozesses von Markoff mit Ent- 
scheidungen und Belohnungen. Drei Beispiele dieser Systeme werden 
vorgeschlagen, betreffend die Bildung von Ketten von Elementen und 
die Wanderung von Elementen in ein- und zwei-dimensionalen Zonen- 
anordnungen. Weiterer Fortschritt wird von dem physikalischen 
Modellieren von unhomogenen Markoffschen Prozessen erwartet. Das 
Anwendungsgebiet der Theorie kann in der Chemie, beim selbst- 
tatigen Kultivieren von Algen und in der Bildung von zufallsbestimm- 
ten Netzen gesehen werden. 


Introduction 

A further development of the theory of automatic control 
results from its application to complexes of many elements 
subject to automatic ordering action. Owing to the impossibility 
of following the dynamics of the very numerous elements of the 
complex, statistical characteristics, accessible to macroscopic 
measurement, may be used for the description of the evolution 
of the group of elements. These characteristics are to be 
compared with the corresponding theoretical ones, derived 
from the mathematical model of the process of configuration 
based upon the properties of the elements and upon the condi¬ 
tions of the process, including the influence of the control 
action. The problem of controlling the development of the 
group of elements leads to the introduction of a deterministic 
control of certain frequency functions of events pertaining to 
the formation of configurations. The corresponding mathemati¬ 
cal models use probabilities instead of frequency functions. 

Definition and Basic Scheme 

A system with automatic control of configuration is a 
system with a complex of elements which develops by auto¬ 
matic control towards an assigned state or set of states, charac¬ 
terized by the configuration of these elements. 

During this development one (or more) of the following 
basic operations of configuration, pertaining to the elements of 
the complex, is realized: the aggregation; the orientation; the 
liaison; the arrangement; the connection. 

The general scheme of a system with automatic control of 
configuration is shown in Figure L Here K = the complex; 
F = the formator; S = the measured state variables of the 
complex; A = the acting variables of the formator; P = the 
perturbing signals acting upon the complex; V = the output 
variables of the complex; R = the command variables. 
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The function of the formator is to elaborate the acting 
signals for influencing the configuration of the elements of 
the complex. The output variables V of the complex are, in 
general, different from the measured state variables S which are 
chosen so as to inform, by their totality, on the configura¬ 
tion of the elements of the complex. 



The index m applies to the intermediate state between the 
initial state (index v) and the final state (index c) and where the 
differences of configurational entropy are 

AS j m = k log ( Z m — Z c ) (3) 

AS fv = k log ( 7 V ~~ Z c ) (4) 

where k is a scale factor. Z* is the number of possible ways 
of having the elements ordered at the initial state, Z m is the 
number of different ways of ordering the elements suiting 
the definition of the intermediate state at a certain phase of 
development and Z c is the number of different ways of ordering 
the elements suiting the requirements upon the final state. 

The configurational entropy is a concept used in statistical 
physics. In crystallography, the entropy change for a transition 
in the crystalline phase is divided into: the change of the 
configurational entropy and the change of thermal entropy 8 . 
The configurational entropy of the arrangement of atoms in a 
lattice is determined by the number of different ways in which 
the atoms may be arranged over the available number ol lattice 
sites. In chemistry, the information content l t of a protein is 
divided into: I s depending upon the amino acid sequence, and 
I c depending upon the configuration of the polypeptide chain 
in the native molecule 9 . 

The state of the complex may be described by different 
concepts and measures depending upon the basic operation 
of configuration of the elements 6 ' 12 . 


Description of the State of the Complex 

An approach to the description of a complex with a great 
number of elements consists in its division into equal zones and 
in considering the group of elements contained in each of 
these zones. The interrelation of the groups contained in the 
zones, especially in the neighbouring ones, may be of interest. 
In two-dimensional representation we draw the meshwork of 
zones as, for example, in Figures 7 and 8. As it may not be 
possible to measure the state variable in all these zones, we 
introduce sample zones . The measured state of the complex 
may be expressed by the measured state variables S in the 
form: (a) of a column vector with elements s 1 (t) 9 s 2 ( t ), ...,s n (t); 
( b ) or, in the case of a two-dimensional arrangement of measur¬ 
ing points, which may be advantageous for the expression of 
the configuration, in the form of a quadratic matrix ||%(*)|| 
(z,y = 1, 2,..., n); (c) or, in the case of a three-dimensional 
arrangement of measuring points, in the form of a cubic matrix 

(0|i (i,j,k=l,2,...,n) (1) 


Quantitative Expression of State During Aggregation 

The simplest expression of the state is in terms of the number 
of elements or of their concentration in the different zones. The 
information connected with the concentration into a single 
zone s' of elements of a certain type i, which previously have 
been distributed over the whole complex, is 

L«-log a (g) [bit] (5) 

where c is is the concentration of the elements of type / in the 
zone s , 

c ik is the concentration of the elements of type i in the 
whole complex. 

When several types i (/ =» 1,2,..of elements are involved, 
we have 

/c=2>g 2 (^-) [bit] (6) 

i \ c lk/ 


Measurements of state variables in three-dimensional or in 
two-dimensional arrays of zones of the complex can be reduced 
by scanning to a sequence of measurements. Similarly, one can 
express the required state of the complex using the command 
variables R. 


A theoretical measure of the state of the complex with many 
elements is the configurational redundancy 


where A S fm < A* S fv . 


R/m — 1 ” 


ASf m 

A S fv 


( 2 ) 


Quantitative Expression of State During Orientation 


The orientation of the elements of a zone of the complex may 
be expressed by an angular measure. The information connect¬ 
ed with the orientation of the ith element of the complex may 
be expressed by its orientation information ! or yy If the configu¬ 
ration of the complex requires that the orientation of the ith 
element be fixed within A<9, A <j>, At//, where 0,</>,t// are Euler 
angles, we have 


1 or [i] : 


=log 2 


87t 3 


A0 i A(/) i Ail/ i 


[bit] 


(7) 
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Quantitative Expression of State During Liaison 

Consider the operation of liaison of the elements in a 
complex of constant volume, with elements of different types i, 
where i = 1, 2, k and denote n l9 zz 2 , n k the numbers of 
these elements. They move and combine at random to form 
new types of elements by liaison. To characterize the develop¬ 
ment of the state of the complex use the probability P ( n, t ), 
which is the probability that in time t the complex has the 
composition n, where n is a vector, whose components are the 
numbers of the elements of the different types. By the action 
of the formator one wishes to influence the probability P («, t). 

Quantitative Expression of State During Arrangement 

The number of sites or lattice sites correctly occupied by 
the elements of the complex is a simple measure of arrangement. 
The information connected with the position of the zth element 
in the complex of volume V may be expressed by its placement 
information 4 ^3. If the configuration of the complex requires 
that the zth element remain within a space Ajq, A y h A z h we have 

/p[i] = l0g2 {^;A^Az} [bit] (8) 

Quantitative Expression of State During Connection 

As a characteristic quantity of a random net, Clark and 
Farley have used the connectivity. An element, z, is connected 
to an element j with a probability P €j which may depend upon 
both z andy and on other characteristic quantities of the net as 
a whole. Uttley has considered the probability of the connection 
of an element in a given position to an input point. This proba¬ 
bility is a function of the position. Beurle has used a proba¬ 
bility of connection of an element to all elements which are in 
a distance r from it, this probability being a function of the 
co-ordinates x, y , z of the element and of the distance r. 

As we are interested in characterizing the state of a develop¬ 
ing random net, the use of test impulses, applied in points of 
sample zones and the measurement of the number of elements 
activated at a given distance in a given direction, or of the speed 
of the signal spreading, can be suggested, according to the 
particular case. These quantities are related to the statistical 
characteristics of the random net. 

The Interaction of the Formator and of the Complex 

The acting variables of the formator are from the point 
of view of automatic control the output variables of a multi¬ 
dimensional controller with many inputs, some of which are the 
measured state variables of the complex. The information about 
the behaviour of the complex only from the measurement of 
external input and output variables of the complex would be 
insufficient. This is also in compliance with the principle of 
uncertainty in the structural behaviour of multivariable systems, 
formulated by Mesarovic 1 . 

Therefore direct measurement of the state variables, amend¬ 
ed by theoretical relations yielded from the mathematical model 
of the configuration process, is required. 

A methodical approach towards the identification of the 
process occurring in the complex consists in the following 
stages: 


(1) The formation of a mathematical model of the process 
of configuration of the elements. 

(2) Computation of the relevant mean values S( T ) ( t ) on a 
mathematical machine on the basis of the mathematical model 
and using complementary information about the physical 
conditions of the process. 

(3) Comparison of the computed mean values S ( T ) and their 
development in time with the measured state variables S. 

(4) The appropriate adaptive correction of the model under 2 
in order to minimize the difference of the comparison under 3. 

The objective is to obtain an approximate model of the 
process in the complex based on theoretical results from the 
statistical dynamics of the elements of the complex. Such a model 
is intended to bring a better insight into the mechanism of 
configurational changes and to help in choosing the method 
of control. 

Two basic deviations which have to be considered in the 
theory of these systems are: 

(1) <P(t)—R(t)—S(t) (9) 

where R ( t ) are the command variables and S (t ) the measured 
state variables, 

(2) <p m (t)=S m (t)-S(t) (10) 

where S( r ) (t) are theoretical state variables computed from 
the mathematical model. The minimization of <p( T ) (t) is a form 
of the identification problem of the process in the complex. 

When using the cubic matrix expressions the basic devia¬ 
tions are: 

(«) ||<M0|HlM0lHkv*(0|[ (ii) 

(b) liCCOlhll^COll-II^COll (i,j,k=l,2,...,n) (12) 

Consider a set of discrete simultaneous values of the time 
functions. A typical operation with the cubic matrix consists 
in applying to the trilinear form 

F = E 4>ijk y>s Zk (13) 

i, j, /c = 1 

the linear transformation 

n 

Xy=Y Syi X i = (14) 

1=1 

with the quadratic matrix 

g=lkJ (y,i = l,2,...,«) (15) 

There is then obtained the product cubic matrix in the index z: 

00} 2=11 Z 4>jjkg r i\\ (16) 

y = l 

Similarly, 

00} HI I hjtgvW (i7) 

7=1 

0{ fc }g=llE <PinSyic\\ (18) 

7 = 1 
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Applying certain bilinear transformations with a cubic 
matrix of /zth order to the trilinear form we would get product 
tetric matrices of the /zth order, which are already more com¬ 
plex to handle 3 . 

Owing to the number of input variables of the formator and 
to the complexity of its function, the formator should be a 
digital computer. 

A special role in the introduction of this point of view of 
influencing probabilities of events by the formator is to be 
assigned to the concept of controlled probabilistic transducers. 



Figure 2 

As an example of a type of controlled probabilistic transducer 
let us derive from the transducer type, mentioned by Kochen 11 , 
the transducer of Figure 2, where the conditional probabilities 
c x and c 2 are, respectively, functions of the acting variables 
a l9 a 2 of the formator. The conditional probabilities are 


Ci=P|>(0=l 

|xO-l) = l] 

(19) 

c 2 =T[>(0 = 1 

x(l —1)=0] 

(20) 


where x (t) is a two-valued variable (value 1 or zero), and 
t denotes the number of the time interval. 



Figure 3 

Another type of controlled probabilistic transducer {Figure 3) 
can be derived from the concept of binomial probabilistic trans¬ 
form, studied by Sugimori 4 , by making the parameters p and 3 
functions of the acting variables a x and a 2 of the formator. Let 
x, y be the numbers of signals of value 1 and 3 a time lag. 
For the special case, when p and 3 are constants, and when x has 
a Poisson distribution with parameter l 

p(x)=^e~ A (21) 

then in time 3 and only in this time 

CO / 2 7)V 

p(y)= t, p(x)b(y;x,p)=±— r e~ lp (22) 

x=o y * 


where h (y; x, p) is the binomial distribution of the random 
variable S x denoting the number of successes in x Bernouilli 
trials. 

Further, a simple scheme of a controlled probabilistic 
transducer without time-lag for a sequence of equidistant pulses 
can be represented by a logical product element, steered by a 
generator of random pulses with controlled statistical parameter, 
under the assumption of the synchronization of the pulse 
sequences. 

The Optimization of the Process of Configuration 

The attainment of a certain state of configuration (or set of 
states) can be considered as a result of two opposite actions: the 
one of configurational ordering, the other of disordering. 

An important problem in connection with the systems with 
automatic control of configuration can be raised: the stability 
of the controlled complex in remaining in a definite set of 
states. For the description of the development of some multi- 
variable complexes, mathematical models based on Markov 
processes seem suitable. 

The number of states which the multivariable complex can 
take is infinite and countable. The theory of Markov proc¬ 
esses with an infinite and countable number of states is to 
be applied. 

In order to formulate certain basic relations concerning the 
optimization of the process of automatic configuration we shall 
purposely limit ourselves to consider it as a Markov process with 
a finite number N of states. Let this process be defined by the 
matrix of transition probabilities P — [p jk ] and by the matrix 
of rewards W = [w jk ], where the indexes j and k apply to the 
transition from the state j to the state Ic. 

The result w jk is the increment of configurational ordering 
associated with this transition. It is to be expressed in units 
of configurational measure. A physical interpretation may be, 
for example, the increase in the number of a new type of 
element formed by the liaison of two types of elements, or of the 
number of correctly occupied sites in a lattice etc. During the 
development of the complex some w jk can be negative. 

Applying the method of Howard 2 , one expresses the mean 
reward from a transition 

( 23 ) 

;=i 

where = the probability of the complex being in state j after 
a large number of steps, 

q j = the immediate reward expected at state/, i.e. the 
expected reward connected with the transition of 
the complex from the state / to the next one. 

The immediate result q s is 

Qj = E Pjk^jk O' = 1,2(24) 
/<= 1 

The aim is to maximalize the mean reward g of the Markov 
process of configuration. 

Supposing that at each state j one of the alternatives of the 
action of the formator upon the complex can be chosen. To 
each alternative a corresponds a transition probability p? k and 
a reward w %. When chosen, the alternative becomes a decision d. 
One denotes by dj (n ) the decision taken at the state /, which 
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means n states before the attainment of the final state. A column 
vector d 9 with elements dj (n) expresses the chosen policy. The 
total expected reward during the development of the complex 
in n steps starting from the state j and applying a specific policy 
is Vj (ri). Under the assumption of the Markov process being 
completely ergodic, we have for large n 

V j(ri) = ng+Vj (j=l,2,...,N) (25) 

Between the introduced quantities there is the relation 

N 

g + Vj = qj + Z Pjk v k 0 = 1,2,. ..,N) (26) 

k= 1 

Following further the method of Howard, let v N = 0 and call Vj 
the relative values of the policy. By a judicious choice of the 
Pjh and q 6 for each state; the reward g is to be maximalized. 

(1) For each state; the alternative a r which maximalizes the 
value [see the relation (24)] 

<fi+ I P>k (27) 

1 

is to be determined. Here the index a denotes the values belong¬ 
ing to the alternative a. Then by putting p“' k = p jk ; q a j = q j9 
the resulting values are used below. 

(2) Using these p jlc and q$ in the system of linear simul¬ 
taneous equations (26) and by solving this system one gets the 
Vj and the g which will be again used in (1). By the iterative 
computation process involving (1) and (2) one finally gets the g, 
and the p jk and qj. The required speed of computation would be 
high. On the other hand, assuming that the complex evolves 
relatively slowly, the change of the p jk would be done by the 
action variables of the formator. Without the action of the for- 
mator, the isolated complex would develop ‘spontaneously’ with 
transition probabilities pfy. 

Theoretical investigations require the application of the 
theory of Markov processes with an infinite and countable 
number of states. Some notions are common with the theory of 
Markov processes with a finite number of states, as for example, 
the notion of undecomposable groups, of transition groups and 
of final groups. 

Suggested Examples of Systems 

As an example of a system with automatic control of con¬ 
figuration, consider a system with controlled operation of liaison 
of three different types A, B, C of the very numerous elements of 
its complex. The elements move with random Brownian move¬ 
ment and have the following properties: (/) when A and B collide, 
a new element D results, (ii) when D and C collide, a new 
element E results, (iii) the collisions of the elements are at 
random, (iv) the liaisons are irreversible, (v) the direct liaison of 



C with A or B is impossible, (vi) the liaison rate parameters are 
k± and k 2 (Figure 4\ 

The state variables of the complex at time t = 0 are n a0 , 
n bo, n c0 , 0, 0—the numbers of elements of the different types. 
The command variables r l9 r 2 are the required numbers of the 
elements D and E respectively (Figure 5). The acting variables of 
the formator are a l9 a 2 , influencing the liaison rate parameters 
kj and k 2 respectively. Let x l9 x 2 be the number of elements D 
and E respectively at time t. 

The control of the operation of liaison is based on making 
the liaison rate parameters appropriate functions of time: 
K (t), k 2 (t). 



Figure 5 


In order to simplify the expressions, first consider k x and k 2 
as constants in the mathematical model. The differential equation 
describing the development of the complex is then: 

dP(x u x 2 ,t) . , 

~ (n a0 -xj (n b o - x t ) P (x ls x 2 , t) 

+ k 2 (x 1 + 1) (n c0 —x 2 + l)P(x 1 + l,x 2 -l,t) 
~ k 2 (xj)(n c o — x 2 )P(x 2 , x 2 , t) 

(28) 

Using the method of the generating function one takes 

00 

F(s u s 2 ;t)= £ P(x u x 2> t)s*‘ s 2 2 (29) 

•VI, *2 = 0 

and finds 
dF 

-Qf n a0 n b0 (1 ~ s l)] 

dF 

Sl ^~~ Sl ^ n aod-n b0 — l) 

+/c 2 [(^i — 52 ) —w c0 ( 5 i “5 2 )]} (30) 

The boundary conditions are 

F(0,0;0) = 1 and F(l } l;t) = l 
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The mean values of the numbers n d and n e of the elements D and 
E respectively are then as functions of time 

? 7 V,(0 = =^- log F(s 1 ,s 2 ;0 (31) 

L Ui l J Sl =S 2 = 1 

(0 = = 7 - log F (s t , s 2 ; 0 (32) 

L U ^2 J 51 =S 2 = 1 

The dispersions are 


4 ( 0 = 


g 2 q *” 

?logF(s 1 ,s 2 ;t)+— logf(s 1 ,s 2 ;0 ' 

J Si=S 2 =l 


L 9s i 


Q s x 

~ 02 0 “I 

4(0 = g 7 | l 0 S F(h,s 2 ; 0 +Q^ log i ? (S i,s 2 ;oj _ _l 


(33) 

(34) 


Thus the model of the complex may be represented as a 
black box with 2 inputs: k x and k 2 , with an initial state, charac¬ 
terized by the vector components n a0 , n b0 , n c0i 0 , 0 at time t = 0 , 
and with 2 outputs: m nd (t), m„ c (t) or a 2 nd (f), a 2 ne (t). 

More generally, if k x and k 2 are not constants, set in advance, 
but change in time under the control action, the corresponding 
Markov process is non-homogeneous. 



Figure 6 


On Figure 6 there is a closed oriented chain of reservoirs B l9 
B 2 , # 3 , containing many elements of the same type. This chain 
forms the complex. The acting variables of the formator a l9 a 2 , a z 
control the probability transducers represented by full points. 
The probabilities of the random transitions of elements from one 
reservoir to another are thus controlled. The aim of the function 
of the system is to reach a repartition of the elements over the 
reservoirs, prescribed by the command variables r a , r 2 , r z . The 
total number of elements is N. The number of elements contained 
in the reservoir B 2 at time t is x 2 . The probability of the transition 
x 2 -> -f 1 in the time interval ( t , t -f At) is 

X X2 At + o (At) 

where, at first, let the 2 in the relation 

4 2 = A -* 2 (35) 

be constant in time. 


If at time t the reservoir B 2 contains x 2 elements, the prob¬ 
ability of the transition x 2 —1 in the time interval 

(t, t + At) is . ,. . 

H X2 At+o(At) 

where, at first, let the fi in the relation 


(36) 

be constant in time. 

The probability of the transition to a number of elements 
other than x 2 + 1 or x 2 — 1 is o (At). 

The probability of no change in the time interval (i t , t + At) 

l-(4 2 +^ 2 )Ai + o(A t) 


By making, in addition, similar assumptions for the reser¬ 
voirs B z and B l9 one gets the system of differential equations: 


dP d ^^ - =4,-i P Xi - 1 (0 - (4,+a**,) P Xi (0+a»x«+ 1 p x ,+1 

for x f =l,2,... and i = l,2,3 


( 0 , 

(37) 


When the parameters 2 and gt in relations as (35) and (36) 
change in time, under the control action of the formator, one 
has to deal with Markov processes of the birth and death type 
non-homogeneous in time, describing the development of each 
of the reservoirs. 

Another example of system may be suggested with com¬ 
plexes whose schematic representation is in the form of a two- 
dimensional array of zones, which may have, for example, 
a rectangular (Figure 7) or a triangular (Figure 8) form. The zones 
contain many elements. The transition of the elements from one 
zone to other zones is controlled by probabilistic transducers 
steered by the acting variables of the formator and represented 
as full dots. On Figure 7 the state of the selected zone R 22 is a 
function of the states of the neighbouring zones. Considering at 
first a Markov process homogeneous in time as a model of the 
development of the zone R 22 , which can be represented as a 



Figure 7 
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rectangle with two inputs and two outputs, one makes the 
following assumptions: 

The number of elements in the zone R 22 at time t is x 22 . 

The probability of the transition x 22 -> x 22 + 1 in the time 
interval (t, t + At) is 

^X 2 2 s At 4 -X X22z At 4- o (A t) 

The probability of the transition x 22 -> x 22 — 1 in the time 
interval (t,t + At), if at time t the zone is in state x 22 (x 22 = 1 , 

2 ,...), is . , x 

fi X22j At+/u X2:!ii At+o(At) 

The probability of the transition to a state other than 
x 22 + 1 or x 22 — 1 is o ( t). 

The probability of no change of state is 

1 (A*22s A«22z Px,22x) ^ ^ (AO 

The corresponding Markov process pertaining to the zone 
R 22 is of the birth and death type. 

Because of the interrelation of the zones there is an inter¬ 
dependence between the parameters X and y relative to neigh¬ 
bouring zones. It may be e. g. 

A X22s = l i X 1 2j‘ X l2 l f i X22j := ^X 3 25' X 32[ 

^X22z~l i X21v' X 21 l f i X 2 2v == ^X232' X 23 J 

The quantities ju ikj and y ihv , where /, k = 1, 2,..., n, may be 
arranged into quadratic matrices. Owing to the action of the 
probabilistic transducers, the ju ikj and y ikv change in time. 

Prospects of Development of the Theory and of its Applications 

There is a large field for the development of the theory of 
systems with automatic control of configuration. The methods 
and results of the statistical mechanics form the basis for the 
dynamics of multivariable complexes. The representativeness 
of mathematical models is to be checked against physical 
measurements. The solution of problems, related to the Markov 


processes of configurational development non-homogeneous in 
time, could be aided by modelling the process on simulators 
using random signal generators. 

The field of application of the theory may be seen, for example, 
in these directions: the influencing of the formation of strips of 
molecules; the automatic control of the cultivation of micro¬ 
organisms, such as algae; the formation of random nets. 

Fruitful suggestions from Professor Robert Fortet of Paris and 
from Professor Jaroslav Kozesnik of Prague are gratefully 
acknowledged. 
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DISCUSSION 


J. Sklansky, RCA Laboratories , Princeton, New Jersey, U.S.A. 

The possibility of using Markov chain models to gain insight into 
configuration-controlled processess—or, equivalently, ‘self-organizing’ 
processes—is attractive, because the theory of Markov chains is so 
well developed. The examples in the paper demonstrate that these 
models offer a way of analysing processes involving large numbers 
of homogeneously distributed elements in one or a few enclosed 
volumes. 

Unfortunately, these models do not seem so promising for random 
nets. In random nets we generally cannot find a useful division of the 
net into ‘zones’, as suggested in the paper, because in a random net the 
coupling among the zones usually will involve much more than simple 
neighbourhood coupling. This will be true, for example, in an ‘ele¬ 
mentary perceptron’ 1 , in which each sensor unit (‘S unit’) is randomly 
connected to a subset of the set of associator units (‘A units’), with no 
favouring of nearest neighbours. 

Another obstacle to using Markov chain models for random nets 
is the high dimensionality of the resulting Markov chains. To demon¬ 
strate this, consider the very simple perceptron shown in Figure A. 
The reinforcement signal e adds an increment r] to either or both of 
the gains k x and k 2 . If the stimulus is a sequence of statistically 


independent vectors (n), s 2 Qi)}, and if we define the state of the 
system as the vector {k l9 kfs, the Markov chain model of the system 
has the form of a two-dimensional random walk, as shown in Figure B. 


Stimulus 

vector 


Random 

network 



Figure A. A very simple perceptron with error-corrective reinforcement 


662 


ON SYSTEMS WITH AUTOMATIC CONTROL OF CONFIGURATION 


I I 
I I 



Figure B. A two-dimensional random walk 



If each k% ranges over N values, the number of states in the Markov 
chain model is N 2 . In more realistic perceptrons, however, the number 
of A units is large—100 or more. In general, the Markov chain model 
will be an ot-dimensional random walk with N* states, where a is the 
number of A units. If N = 10 and a = 100—these are very conservative 
estimates— the stochastic matrix of the Markov chain obviously cannot 
be held in the memory of any real computer. 

One approach which may overcome this difficulty is through a 
study of the eigenvalues of oc-dimensional random walks 2 . Another 
approach is to analyse only the convergence properties of the process 
as the time, n , approaches infinity 3 . 

A third approach is to view the entire proceses in the manner of 
mathematical psychologists toward human beings: i.e., partition of the 
externally observed behaviour patterns into a few states. A simple 
example of such a partitioning is shown in Figure C. Here the entire 
process is modelled by a two-state Markov chain, even though an 
internal examination of the system or ‘organism’ will most likely 
yield a many-state Markov chain. In the simple two-state model in the 
figure, one state corresponds to a ‘successful’ behaviour pattern; the 
other state to an ‘unsuccessful’ behaviour pattern. 

We see, then, that a major problem in the study of self-organizing 
random nets is the modelling of certain classes of many-state Markov 
chains by few-state Markov chains. As yet, little attention has been 
given to this problem. But the interest in it will grow as the population 
of artificial self-organizing systems grows. With the advent of this 
new form of ‘population explosion’, engineers will take on more of 
the role of psychologists, and, we believe, will tend to describe the 
behaviour of self-organizing systems by few-state Markov chains. 
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J. Benes, in reply 

I agree fully with Mr. Sklansky that the major problem in the study 
of self-organizing random nets is the modelling of certain classes of 
many-state Markov chains by few-state Markov chains. However, 
I would like to express my opinion on the direction of research, 
and to deal with both difficulties pointed out by Mr. Sklansky. 

One way seems to be to consider the formator as a system which 
is in part organized in levels, these levels being hierarchically linked. 
In such a case it seems to me that the concept of the division of the 
complex into zones could be maintained and could be useful. Such 
a possibility can be seen, of course, if we consider random nets not 
specifically for perception but for the study of systems of organization, 
as in economical systems, distribution nets etc. 


R. Tar JAN, J, Ponty-utca 2, Budapest , Hungary 

The problem of configuration-controlled or self-organized systems 
is indeed a very exciting one, and rightly deserves great interest. The 
introduction of probabilistic transducers as a method of forming zones 
in random nets seems also a necessary step in attaining a model of a 
living organism such as the central nervous system. However, looking 
at the configuration controlled systems as a possible mathematical 
model of the central nervous system, it seems to be necessary, that 
besides the five basic operations introduced by the author, a sixth 
operation be introduced, namely that of the separation. This would 
be clearly an analogon of the biological inhibition which, as is well 
known, plays a dominant role in the proper functioning of the brain, 
e. g. in the selection of memory contents. 

One further remark concerns the measure of order given by the 
author in (2). This measure takes into account only the numbers of the 
effective respective possible connections. Now a complex such as the 
central nervous system is not only connected, but also has a very 
definite organization necessary for survival. Intuitively it is clear that 
the organization is a selected set of connections, the selection being 
made so as to suit a given task. It is also intuitively clear that, given 
the task, there are many possible organizations, one of them being 
more complicated than the others. So, in dealing with models of such 
complexity, the problem of having rigorous definitions of the notations 
of organization arises, such as the complexity of organization with 
respect to a given task. The situation is analogous to an optimizing 
system which can be optimized only with respect to a well-defined 
criterion. I would be obliged if Professor Benes will kindly comment 
on this problem. 


J. Benes, in reply 

Dr. Tarjan’s first question concerns the selection of zones. I think this 
is a question of applying some mathematical statistical methods to 
select the number and the place of the zones which will be used for 
measuring the state of the complex as a whole, to have zones selected 
which would be really representative. 

The first method is mentioned in the paper. A second method 
which now seems very promising is the use of the well-known technique 
of fibre-optics, and using this technique it would perhaps be possible 
to solve this problem of selection of zones. 

The second question is whether or not to use redundancy in the 
measuring of organization. I would say that the concept of configura¬ 
tional redundancy seems to be important from the point of view 
of theory. In order to be able to use certain concepts of information 
theory in my paper I have already pointed out that there are very 
simple and, for technical use, important criteria of ordering and 
measuring; for instance, the number of the occupied sites in a lattice 
or the number of elements of a certain kind and type which had been 
formed by reaction. 
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Approximation of Industrial Control Systems for 
Optimized Non-linear Control with Restricted Rate of Correction 

Using Conventional Controllers 

E. PAVLIK 


Summary 

The positioning speed of servo-motors for industrial process control 
systems is always limited. With control loops of low inertia this speed 
limitation has a high influence on the control behaviour and causes a 
considerable non-linearity of the control loop. According to a theory 1 - 3 
for systems with limited parameters it has been proved that the 
optimum transient response for a discrete disturbance and the 
optimum transient responses for different disturbances are predeter¬ 
mined. Due to the complexity and sensitivity to disturbances of the 
required control equipments, however, the experiences gained from 
the optimalization theory have been of little consequence for practical 
control problems. Thus no rules for design, adjustment and quality of 
conventional continuous controllers of the PI or PID type, providing 
a good approximation to the above transient response characteristics, 
have been formulated. The same applies to the three-position con¬ 
troller (pulse controller) commonly used in practice. For this reason, 
attempts are being made to develop methods giving suitable approxi¬ 
mations to the optimum characteristics. For simple systems, the ad¬ 
justment parameters of the above-mentioned industrial controllers 
can be calculated by an approximation method. 

Sommaire 

La limitation des vitesses de positionnement des servomoteurs indus¬ 
tries, dans des boucles de commande a faible inertie, influence grande- 
ment le comportement de la commande et, souvent, produit une non- 
linearite considdrable. Dans certaines etudes 1-3 , on a predetermine 
la reponse transitoire optimale par rapport a une perturbation dis¬ 
continue et k un ensemble de perturbations diverses. Toutefois, l’e- 
quipement necessaire est a la fois trop complexe et trop sensible aux 
perturbations. De ce fait, theorie d’optimalisation n’a que peu 
d effets dans la pratique ou l’on utilise des regulateurs classiques du 
type PI, PID, a trois positions, etc. Dans ce rapport, on presente une 
mdthode de calcul approximative, permettant de definir les valeurs opti- 
males des parametres reglables des regulateurs mentionnes ci-dessus. 

Zusammenfassung 

Die Stellgeschwindigkeit des Stellantriebs technischer Regelanordnun- 
gen ist stets begrenzt. Besonders bei schnellen Regelkreisen ist diese 
Begrenzung der Stellgeschwindigkeit von groflem EinfluB auf das 
Regelverhalten. Die Theorie 1 ” 3 fur Systeme mit begrenzten Para- 
metern liefert den Beweis, daB fur eine bestimmte Geschwindigkeits- 
begrenzung sowohl der optimale UbergangsprozeB bei einer diskreten 
Storung als auch die Gesamtheit der optimalen Obergangsprozesse bei 
verschiedenen Storungen bereits festliegen. Wegen der Kompliziertheit 
und Storanfalligkeit der dazu erforderlichen Regelorgane haben sich 
die von der Optimaltheorie vermittelten Erkenntnisse bis jetzt wenig 
befruchtend auf die regelungstechnische Praxis ausgewirkt. So existie- 
ren z. B. keine Vorschriften fur den Entwurf, die Einstellung und die 
Beurteilung von konventionellen kontinuierlichen Reglem des PI- oder 
PID-Typs, welche eine moglichst gute Annaherung an den Optimal- 
vorgang bzw. die Gesamtheit der Optimalvorgange gewahrleisten. Das 
gleiche gilt fur die in der regelungstechnischen Praxis weitverbreiteten 


quasistetigen 3-Punkt-Regler (Impulsregler) mit nachgeschaltetem 
Stellmotor. Deshalb wird versucht, Methoden zu entwickeln, die 
brauchbare Naherungen an die Optimalvorgange ermoglichen sollen. 
Fur einfache Systeme lassen sich die Einstellparameter der obengenann- 
ten technischen Regler zur Realisierung optimaler Vorgange mit Hilfe 
einer Naherungstheorie berechnen. 


Theory of Optimized Systems 

The theory of optimum response of a system with one or several 
limited parameters to varying external influences has been fully 
established in the previous studies 1 ” 3 . It is not within the scope 
of this contribution to repeat this theory in all details. However, 
the knowledge of the theory regarding speed restricted systems 
is indispensable for the understanding of the following statement. 
For this reason, a simplified summary of the essentials is given 
here: 

Let us assume a coordinate x$ of a transfer system F (e.g. 
the differential quotient in time of the correcting action) is 
restricted to a given interval. 

x n ^xax i2 ( 1 ) 

In order to obtain the fastest possible change of the output x 
of the system, the coordinate x t has one of the two limit 
values which will control the system in the desired sense. The 
value of the output x and of its derivatives is determined by the 
initial state of the system x A , x A , ... the coordinate x n or x * 2 
respectively and the time t elapsed from the beginning of the 
process up to the particular instant t. 

x=: fo ( x a> x a, x a> • • • > x u or x i2 respectively; t) 

x=z fi ( x a> x a> x a> • • • ? X i % or ^respectively; t) (2) 

x ~ f 2 ( x a> x a> x a> •••> x n or x i2 respectively; t) 

Since the limited coordinate x { is constant at the instant 
when x reaches a given value (e.g. the desired value), the deriva¬ 
tives of x will generally differ from the derivatives of the given 
value, so that no steady state of the system is obtained. In order 
to achieve a steady state at the end of the transient response, 
the value of the restricted coordinate must vary in a definite 
manner. The system responds quickest if the steady state is 
produced by keeping x { in alternating steps at one of its limit 
values, provided that, besides the restriction of coordinate x i9 
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the transfer function F is linear and has real non-positive 
poles 1 ” 3 . 

To this end, the process is split up into a number of intervals 
h, h, A ... During each interval the restricted coordinate a* is 
maintained constant at one of the two limit values. When 
changing over to the next interval, is given the opposite 
limit value, so that it alternates from one limit value to the 
other. 

The order of the (linear) transfer differential equation, 
which relates the output a and its derivatives with the restricted 
coordinate, is indicated by n. The initial condition and the 
desired condition are determined by the output x and its n -1 th 
derivative. 

If F is, for instance, a guided system and the governing 
variable is Z x , eqn (2) becomes: 

x = Z 1 = / 0 (x A ,x A ,..., t u t 2 ,...) 

X = Z>i=fi (x A , X A , A A •*•) ^ ^ 

’(»- 1) (»-l) 

x = Z x =f n ~.i (x A , x A9 ..., A t 2 , • •*) ^ 

( 3 ) is a family of n equations and, in order to satisfy the n con¬ 
ditions, at least n independent parameters must be available, 
e.g. the durations of the intervals t l9 t 2 . ... In this case, n is the 
minimum number of intervals required to bring the system 
from the initial condition to the desired condition. The family 
of equations has, therefore, n unknowns, t l9 t 2 ,..., t n , the n inter¬ 
vals, which can be determined as the roots of the family of 
eqns (3). 

Equation (3) can also be regarded geometrically. Con¬ 
sidering Z 1 as time function Z x {t), eqn (3) represents the 
necessary and sufficient condition to accomplish that, after the 
last interval t n has elapsed, the output x(t) is identical with the 
input Z x (r), because x(t) and Z a (t) are coincidental functions 
up to the /zth order, provided the condition of reproducibility 
[eqn (4)] is fulfilled. 

This is the case if 

[fl B Zf ) + a ll _ 1 zS , "’ 1) + ...a{Z^a 0 Z{\SK (4) 

where K is the maximum value of the restricted coordinate. 
Furthermore, the differential equation of the transfer system F 
must have the parameters a n9 a n - l9 a 0 . 

If, for example, Z x is a step function of the amplitude Z u , 
the n optimum switching instants (/ = 1 ... ri), corresponding 
to the nth order of the system, can be calculated from eqn (3). 
The switching instants are functions of the system parameters 
and of the step magnitude Z n . Generally, for an input Z l9 the 
switching instants are functions of the system parameters and 
of the parameters of Z x . If within the intervals found in this way 
the restricted coordinate is alternately kept at one of its two 
limit values, the variable x will reach the desired condition 
aperiodically and in the shortest possible time. Optimum switch¬ 
ing instants are frequently obtained as a result of automatic 
control actions by feeding back the output a to the input of the 
restricted coordinate. 

At this juncture it is of particular importance to introduce 
the term ‘completeness of the optimum processes’, formulated 
by Solodovnikov 3 . 

If a system is said to qualify for ‘completeness of optimum 
processes’, it means that optimum switching instants and thus 


optimum responses are automatically achieved not only for 
one discrete external influence, but optimum transfer condi¬ 
tions exist in such a system for a whole family ot inputs, e.g. 
step disturbances of any amplitude or ramp disturbances of 
any slope or both kinds of disturbances at the same time. This 
is much the same as with linear systems where, once the correct 
adjustment for a certain kind of input has been found, this will 
also hold good for any amplitude. This amazing fact can be 
explained as follows: 

Let us assume that for an arbitrary external influence Z 
(variation of desired value or load change) the optimum 
switching instants have been calculated from eqn (3). As pointed 
out above, these switching instants arc functions of Z: 

t 2 = (p 2 (^) (^) 

t n = <Pn ( Z ) 

The transient responses of the output x and its derivatives are 
functions of these intervals: 

— f Oil ^2 J'.'jO 

*(0==/(A (6) 

(n-l) («~ D 

* = / (A A—A,) 

Elimination of all interval times from the equation system 
(6), would enable relationships to be established between a* and 
its derivatives, independent of the interval times. If it is, for 
example, assumed that it is possible to eliminate all interval 
times from the equations for x(t) and a (A as can indeed be 
done in the case of some simple systems with practical applica¬ 
tions, the equation takes the form 

x = <p(x) (7) 

and is, therefore, not dependent on the interval times. Equa¬ 
tion (7) can be re-arranged to 

x — (p (x) = 0 (H) 

The system yields the output a% which is transformed to 
cp(x). The latter is subtracted from x in accordance with eqn (8) 
and the difference is fed back, as a compensating signal, to the 
input, namely the summing point of the external influence or 
the restricted coordinate respectively. Obviously, this will pro¬ 
duce an effect only if the process is not an optimum. Expressed 
in terms of control technique, the purpose of the feedback is 
to bring the actual process nearer to optimum conditions, since 
for the optimum process the relation (8) is under all circum¬ 
stances obligatory. 

Frequently it is not possible to eliminate completely the 
interval times from (6) without involving higher derivatives. In 
this case, the application of the function transfer (p (a, a, .. .), 
found from eqn (8) becomes impracticable, owing to the in¬ 
compatibility of higher derivatives with the noise level. Further¬ 
more, higher derivatives arc mostly variables of transcendental 
functions, which cannot be made use of in practice at a reason¬ 
able expense. 


665 


E. PAVLIK 


Theory of Quasi-Optimum Systems 

Approximation via the Optimum Switching Instants 

If the difference between the desired value and the actual 
value is x w , the n optimum switching instants t l9 t 2 , ..., t n can 
be determined by the following rearrangement of the family 
of eqn (3): 


0 — (fn) — / 0 

(x A , X A , . 

->X ( a 


* * 0 

11 

X* 

£ 

1 

(x A , X A ,. 


(9) 

A— v (»“ 1 >_ f 
u ~ x w (t n )~ In- : 

1 (x A , x a , ■ 



Assume that 

the t x ... 

. t n have been determined for 

one 


particular external influence Z (variation of the desired value 
or load change). For a system of the nth order the 
U (z = 1— /z) are functions of the external influence and, 
furthermore, according to eqn (9), of the initial values x ( j } 
[i = 0 ... («-l)] and of the balance values x%\ tn) [i = 0 ..,(«-l)]. 

Now assume that the correcting servo-motor together 
with the controlled plant form the system of the nth order 
and that the correction speed of the servo-motor is limited. 
This speed-limited servo-motor element with the maximum 
speeds of operation +K 0 (forward) and — K 0 (reverse) can be 
considered as an integrator with integration time constant 1, 
preceded by a restricting element, having the saturation values 
+i^ 0 a nd —K 0 (Figure 1). 

It follows from eqn (9) that the restricting or relay element, 
preceding the motor, must change (n~\) times in the intervals 
h ... from the one extreme position to the other, in order to 
correct the deviation x w and all its (n- 1) derivatives within the 
shortest possible time. 

Figure 2 shows the block diagram of a system incorporating 
a controller R of a certain type [e.g., a conventional continuous 
proportional-plus-integral (PI) or proportional-plus-integral- 
plus-derivative (FID) controller] which may have K free setting 
parameters. 


Relay or 
saturation 
element 



z 1 =Governing variable 
22 =Disturbing variable 
x s =Control variable = sum total 
of all variables acting upon 
the restricting element 


Figure 1 



x s = Yi (Controller output) 
Figure 2 


If the correcting servo-motor has a static proportionality 
effect (steady-state offset caused by rigid feedback), the block 
diagram of the system will slightly change as indicated in 
Figure 3. The controller may again have K free setting para¬ 
meters. 

The block diagram, Figure 4, represents an impulse-type 
system with a 2- or 3-position relay as control amplifier. Ru is 
the feedback, having again K free setting parameters. 

Figures 3 and 4 are typical block diagrams for a large number 
of (continuous or quasi-continuous) control systems met in 
practice. 



x s=yi _ y y= Correcting variable 
Figure 3 



x s =Xw-y R 


Figure 4 


The input amplitudes required for moving the restricting 
elements to the end positions are insignificant and can be 
neglected. The condition for obtaining optimum response can 
be written as 

x s (ti) = 0 for z = l,...,rc (10) 

because the servo-motor has to be changed over at each in¬ 
stant t x ... t n . However, must not disappear between switching 
instants t^ so that the restricting element can definitely reach 
the extreme positions. x w can be calculated for any instant 
t x ... t n . 

Since it has been assumed that the controller in Figures 2 
and 3 has K free setting parameters, the output y x of the con¬ 
troller (transfer function) will be: 


y 1 f (Ki) K-29 "'i Kjr 9 X w9 i) (11) 

For conventional continuous controllers, for instance: 

P controller: y x = K x x w 


PI controller: 

PID controller: 


yi = K x x w +K 2 -J x w dt 

JT ^ df+ 
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For the controller in Figures 2 and 3 the number of free 
setting parameters K must equal n in order to satisfy eqn (10). 
One can, therefore, write down, e.g., for the system in Figure 2 : 

x s(h) = yi (A) = 0 = / (K u K n , 

^s(^2) = y 1 fe) == 0 = f *5 12 ) (12) 

~ yX ( Jrt ) —0 = / •••? K ns ti, t 2 , 

A similar equation system can be developed for that in 
Figure 3 : 

x s (0 = Oh ~ y)t, = 0 for i=l,...,n (13) 

in which way n equations are again obtained for the determi¬ 
nation of the n required parameters K. 

It has been assumed that the feedback of the system shown 
in Figure 4 has again K free setting parameters. In this case, 
also, K must equal n in order to satisfy eqn (10) and to obtain 
the optimum response for one disturbance Z 1 or Z 2 . 

Summing up, it can be stated that, in order to obtain the 
optimum response to a particular disturbance Z, the arrange¬ 
ment of a control system of the nth order (including the correct¬ 
ing servo-motor), must be such that at least K~ n free setting 
parameters are available (with the only exception of a controlled 
plant of order 0). The response to any other disturbance will not 
be the optimum. In the case of conventional controllers without 
non-linear computer elements, an approximation to the ‘com¬ 
pleteness of the optimum processes’ (which can be accomplished 
only by non-linear elements) can be obtained by determining 
the optimum parameters for two or several disturbances and by 
interpolating between them. 

Attention must be drawn to one important fact emerging 
from the equation systems for the determination of the K para¬ 
meters: since a real controller has hardly more than three free 
setting parameters, it is not possible to optimize completely 
systems of more than the third order. However, the present 
contribution deals particularly with fast control systems, which 
generally belong to the first and second order, so that one need 
not worry about the above-mentioned limitation. 


Approximation via the Optimum Control Function 

The following relationship between the output x and its 
derivatives exists in an assumed optimized system which satisfies 


eqn (9). 


x = cp(x,x 9 x ,...) 


(14) 


The optimum control function cp is normally non-linear. As 
mentioned above, in order to obtain ‘completeness of the opti¬ 
mum processes’, the control loop must comprise a function 
couverter, which is responsible for satisfying the relationship 


x — cp(x 9 x 9 x, ...) = 0 


at all switching instants f*. 

If this function couverter is too costly, or, in case it is 
necessary, with a view to suppressing small disturbance oscil¬ 
lations, to neglect the higher derivatives or powers of deriva¬ 
tives, an approximation to the relationship (14) must be at¬ 
tempted with the means at our disposal. The same applies to 
conventional P, PI and FID controllers, which are themselves 
primitive computers. A generally applicable approximation to 
the optimum control function cannot be given. An approxima¬ 


tion cannot be successful, unless the influence oi important 
variables is adequately taken into consideration For instance, 
with simple systems, a close approximation can be achieved by 
substituting a linearized element of the form A s jc for frequently 
appearing elements of the optimum control function in the shape 
\l\x It is frequently possible to approximate important con¬ 
trol variables, particularly the higher derivatives, by the use of 
relatively simple means. 


Conditions Under Which Quasi-optimized Systems Can Find 
Practical Application 

If it is intended to approximate by determining the K free 
setting parameters of the system through the knowledge of the 
optimum switching instants, it must be borne in mind that the 
resulting values for the parameters can be practically used only 
if they are physically achievable and if their use would not impair 
the stability of the system. For instance, if eqn (10) yields a 
negative time constant for a feedback, this is obviously useless 
for practical purposes. It is, furthermore, obvious that if a 
calculation of this kind results in a negative value for the total 
gain of a feedback network, this makes no sense at all, since the 
transmittance of a continually negative teedback signal to the 
system input would lead to monotonous instability (unbalance 
of the control amplifier). The same applies, of course, to negative 
proportional bands, etc. Furthermore, the control functions of 
the approximation system, resulting from the determination of 
the K free setting parameters, must be such that the restricting 
element is kept to its extreme positions during the switching 
intervals; in other words, the control function xjt) of the 
approximation system must not disappear between the switching 
instants. This condition can be formulated by: 

x s (0#0 for 1 < 1 „ 

and t j, ( 2 . * * ■. 

t n = instant of the last switching operation because accord¬ 
ing to eqn (10): 

X s (f;) = 0 <=1,...,H 

Moreover, the deviation from 0 must be sufficient to secure 
that the limit values are definitely reached, which can be ex¬ 
pressed by: 

for ( = (16) 

For the stability of the nil positions of an approximation 
system consisting of linear transfer elements except the restrict¬ 
ing elements, the conditions previously outlined 1 are generally 
applicable. To comply with these conditions, the limit system, 
that is the system with an infinitely steep, unrestricted ‘limiting 
element’, must be stable and the transfer function U(t) of the 
linear part must satisfy the following requirements: 

for u (t — 0) — 0 
u(t = 0)>0 

and for u(t = Q) = u (£ = 0) = 0 (17) 

ii(t~ 0)>0 and u (* = G)<() 

This also means that the difference between the degrees of 
the denominator and of the numerator belonging to the transfer 
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function of the linear system part must be ^ 2. The following 
discussions which supplement the full treatment given to the 
subject 1 ’ 3 will be of interest. 

Investigations with an analogue computer have proved that 
with a difference between the degrees of denominator and numer¬ 
ator equal to 2 and compliance with the other conditions, the 
stability is secured in principle, however, that the system becomes 
considerably underdamped when the correcting time of the motor 
is of about the same order of magnitude as the total time lag 
of the controlled plant, which is just the range of interest for 
this contribution. The performance improves considerably if 
it is possible to reduce the difference between denominator and 
numerator to 1. The system then obtains the capacity of as¬ 
suming sliding states (explained in reference 1) which is particu¬ 
larly favourable for the performance of systems governed by a 
variable. 

A correcting servo-motor has a finite correcting speed 
and also a limited correcting action. This constitutes a further 
considerable non-linearity and a serious drawback for the 
quasi-optimization of systems, whether the approximation is 
attempted via the optimum switching instants or the optimum 
control function. Equation (3) allows a decision to be made im¬ 
mediately after calculation of the optimum switching instants, 
whether the limitation of the correcting action will be unfavour¬ 
able to the chances of obtaining quasi-optimization. If the 
system is in equilibrium when the disturbance occurs, the cor¬ 
recting element assumes a certain position within its travel 
range. From this position of equilibrium, it will move towards 
the top or the bottom position, in accordance with the sign of 
the disturbance. Obviously, the first switching interval t t must 
not produce a movement greater than the available remainder 
of the travel range, which is called h , otherwise the travel would 
be forcibly and prematurely terminated by the extreme position 
stop. If Tm is the running time of the motor over its whole 
travel range, the condition just outlined, for unhindered opti¬ 
mum response, can be written 

(18) 

Since t x is always a function of the system parameters and 
disturbance amplitudes, the disturbance amplitudes to be ex¬ 
pected have to be taken into account, which most probably 
involves a statistical problem. Studies carried out by means of 
an analogue computer have proved that the effect of the travel 
limitation mentioned above, dependent on the choice of the 
disturbance parameters, can considerably deteriorate the system 
response, however, without jeopardizing the stability. 

Naturally, the theoretically developed quasi-optimized sys¬ 
tem must be a practical proposition with regard to instrumenta¬ 
tion and must also conform with established control engineering 
requirements. Complex non-linear functions must be excluded 
immediately; this applies also to demands for accurate higher 
derivatives of the controlled condition, demands which, as 
experience has taught, are incompatible with the noise level of 
the system. This obstacle, however, can be overcome by the 
ingenious methods of originating feedback signals respectively 
prior to the controlled plant or the servo-motor. 

Conclusions 

_ From theoretical discussions and the investigations 
with an analogue computer, directions can be deduced for the 
further development of systems and instruments suitable for the 


design of fast-acting automatic controls. These directions call 
for conceptions, which differ from those accepted up to now, or. 
at least, for modifications of such conceptions, particularly with 
reference to fast-acting automatic process control devices. 

At present, almost without exception, PI controllers (mostly 
with fixed proportional band) are applied for fast control, 
particularly flow control, in conjunction with a servo-motor 
having restricted correcting speed. It has become apparent from 
the deliberations outlined above that this cannot be the correct 
solution. If, however, this arrangement has been chosen the 
controller should be given a derivative action, thus considerably 
narrowing the proportional band, particularly with a controlled 
plant of the first order. The closer approximation to the opti¬ 
mum control function thus achieved will result in a greatly im¬ 
proved quality of control. 

It has been realized that it is advisable, for fast control 
systems, to abandon the combination of a linear PI or PID 
controller with a speed-restricted servo-motor element alto¬ 
gether and to pay more attention to relay systems. It has, 
furthermore, been shown that the use of a three-position relay 
instead of a continuous controller, the saturation values of 
which are equal to the response values of the relay, has little 
influence on the transfer behaviour of the system. It follows 
that a pneumatic or hydraulic system conforming to block 
diagram, Figure 4 , will be simpler in instrumentation and better 
in control performance than a system such as that indicated 
in Figure 3 . 

It has, furthermore, been proved that a forced linearization 
of the correction transfer function of speed restricted servo¬ 
motors is inadvisable. In practice, linearization is achieved by 
using a positioner of the PI type for the regulation of the move¬ 
ment of the valve stem thus obtaining correction transfer func¬ 
tions having approximately equal time constants. If, after all, 
a positioner is employed, the restricted co-ordinate, that is the 
maximum available rate of correction, should be utilized to its 
full extent. The ensuing considerable non-linearity of the cor¬ 
recting action should not be corrected by additional devices 
fitted to the servo-motor but by adjustment of the controller 
to the particular non-linearity. 

With reference to the conception of the 3-position con¬ 
troller the following can be said: up to now, the delaying feed¬ 
back, typical for such systems, is normally designed in such a 
way that, with the feedback amplification remaining constant, 
adjustable time constants are available for the charging and 
discharging of the feedback capacitors. This produces an ad¬ 
ditional adjustability of the so-called integral (/) part of the 
controller. Parameter calculations and analogue studies have 
proved beyond doubt that this conception meets the require¬ 
ments of a system with restricted rate of correction only to a 
small extent. For a good approximation to the ‘completeness 
of the optimum processes’ it is not necessary to have unequal 
charging and discharging time constants, but an adjustable feed¬ 
back amplification is of paramount importance. In order to 
clarify this question with absolute certainty it was decided to 
carry out the analogue studies with a linear PC feedback and 
also with a non-linear feedback having unequal charging and 
discharging time constants, the feedback amplification remain¬ 
ing constant (as is the practice of many manufacturers). In all 
cases it was confirmed that it was not possible by this method 
to eliminate, even approximately, the unfavourable effect 
caused by the constant feedback amplification. 
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Finally, it should be stated that this paper has not, to any changes of about a factor of 2 do not have much effect on 

degree, exhaustively dealt with all the problems connected with quasi-optimized systems or on their adjustable parameters, 

the logical and reasonable design of quasi-optimized systems 
or for systems with restricted parameters. The practically References 

always existent non-linearity of the controlled plant had, for , Ya g rw/e der Relaissysteme der automatischen 

instance, to be neglected. The most important of these non- Regelung _ M Unchen; R. Oldenbourg, 1958 

linearities is represented by the non-constant static transfer 2 f ELDB aum, A. A. Optimale Prozesse in Regelsystemen. Munchcn; 
behaviour (gain of the controlled plant). In order that this Oldenbourg, 1962 

property may not pass entirely unnoticed, calculations have s Solodovnikov, W. W. Grundlagen der selbsttdtigen Regelung. 
been carried out for various plant gains, with the result that Vol. 2. MUnchen; R. Oldenbourg, 1959 

DISCUSSION 


Ya. Z. Tsypkin, Institute of Automation and Telemechanics , Kalan- 
chevskaya 15-A, Moscow 1-53 , U.S.S.R. 

The problem of approximate realization of optimal (with respect to 
speed) processes by standard controllers is very important from the 
practical point of view, and in this field the author has obtained very 
interesting results. 

As the optimal system operates in the large deviations mode, the 
condition of steady-state stability (‘in the small’) would not be enough 
in all cases. The steady state of a relay system should be stable ‘in the 
large’. It is easy to write down the stability condition ‘in the large’ in 

the form: ImW*(j(ti)< 0, 0<co<oo (1) 

where W* (jco ) is the frequency response of the linear part of the relay 
system. This condition includes the conditions (17). Perhaps,with the 
exact realization of optimal processes, the stability conditions ‘in the 
large’ will be satisfied. In the approximate realization, instead of con¬ 
ditions (17), it is necessary to use condition (1). 

These remarks are additional to the very rational and efficient way 
chosen by the author for the approximate realization of optimal 
systems by comparatively simple technical means. 

E. Pavlik, in reply 

Many thanks to Professor Tsypkin for his very interesting contribution. 
It is true that condition (1) of his discussion remarks contains con¬ 
dition (17). As to exact optimum processes, eqn (17) is, in my opinion, 
sufficient, since the system at the last switching moment t u comes 
automatically in the state of stability of small deviations, the con¬ 
trolled condition x w and all its derivations being eliminated as far 
as the («—l)th order. I agree with the Professor that condition (1) 
must be fundamentally complied with for the approximation system. 
For the studies carried out in analogue computation it is true that 
eqn (17) has proved sufficient. Professor Tsypkin’s contribution is a 
valuable completion of my work. 

F. Schreiner, Fuchshohl 18-20, Frankfurt/Main, Germany 

In his conclusions Dr. Pavlik proposes to abandon the combination of 
linear PI or PID controllers with a speed-restricted servo-motor for 
fast control systems, particularly flow control systems. He advises 
that more attention be paid to relay systems. 

(1) What is the definition of a ‘fast’ control system, where a con¬ 
troller, as indicated in Figure 4 of the paper, gives better control 
results than a linear controller. In my opinion the quality of control 

depends on the ratio ——where T m is the running time of the 
T$ 

motor for full range, T s the time constant of the plant, and y z the 
movement of the motor which is required to compensate for a disturb¬ 
ance of amplitude z. 

(2) The positioning speed for an error amplitude x w is proportional 
to —1—. With higher order plants, the influence of the restricted 

Xp ‘ Tn j 

positioning speed decreases, as-must decrease. 

Xp * Tn 


(3) The author proposes to optimize for step disturbances. The 
results of his deliberations are not applicable to sinusoidal disturbances. 
Tests on the analogue computer have shown that the frequency range, 
in which the disturbance amplitudes are decreased by control, is 
narrowed with increasing running time. As Westcott pointed out, for 
linear control systems, a better control quality in one frequency range 
is obtained only at a cost of less quality in another. By a restriction in 
the closed loop the control quality in a certain frequency range will 
be adversely influenced. There will be no improvement in any other 
frequency range. 

(4) Changes of the plant gain by the factor 2 will have no effect 
on quasi-optimized systems and their adjustable parameters only if 
the running time of the motor is longer than the time constant of the 
plant. To get better control results, the running time must be shorter. 
In this case the variation in plant gain by the factor 2 will eventually 
cause instability, as is known from the setting rules by Ziegler and 
Nichols. 


E. Pavlik, in reply • 

(1) Quick control systems are understood to be systems with the 
total delay time of the controlled plant being of the order of the 
time necessary for the servomotor to run through the total servo- 
system with a top floating speed. In the studies of analogue com¬ 
putation carried out, the relation T m IT s is about 1/3 to 3. 

(2) I agree with this contribution but not with the linear inter¬ 
pretation in connection with the equation -. The influence of 

Xp * Tn 

the limited floating speed decreases with the increasing order of the 
controlled plant, the total delay time of the controlled plant 
logically increasing with the same. 

(3) This point has arisen because of a translation error in the 
preprint, which has now been corrected. I agree, of course, with 
Mr. Schreiner in that the time lag of a servomechanism is not sufficient 
to correct the total control behaviour of the systems observed. 

(4) The adjustment rules of Ziegler and Nichols refer to the linear 
systems which do not exist here. We are just starting from the fact 
that the floating speed reaches the limiting values. 


W. Bottcher, Wolfgangstr. 56, Frankfurt I Main, Germany 

In his paper Dr. Pavlik has given interesting proposals for approxima¬ 
tion methods for optimal characteristics of industrial controllers 
with speed-limited servomotors. To this the following remarks are 
added: 

(1) In certain controller arrangements with speed-limited actuators 
the quality of control decreases compared with a linear PID controller, 
if the positioning speed required is faster than the maximum speed. 
Due to restricted positioning speed the control system {Figure 4) is 
damped with increasing disturbance amplitudes whereas the damp¬ 
ing ratio decreases if a controller with a positioning servomechanism 
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with restricted speed (Figure 3 ) is used. Therefore, one has to optimize 
the system of Figure 3 for small disturbances and the system of 
Figure 4 for large disturbances. If this is not done the transient re¬ 
sponses of Figures A and B are the result. The setting of the controllers 
and the speed of the motor are the same for the controller arrange¬ 
ments of Figures 3 and 4. The result of restricted speed is an increasing 
delay time and a decrease of the proportional band. Tests on the 
analogue computer have shown that instabilities can occur. 



t t 

Figure A Figure B 


In spite of this the transient response of the controller arrangement 
Figure 4 shows an increase of the delay time corresponding to the 
error. The reset time Tn and the proportional band are constant. 
This can be proved by analytic methods. The damping ratio of the 
transient response increases. 

As an example, in Figures C and JD can be seen the transient 
responses of the closed loop systems for step changes of desired value. 
The controllers used for the test on the analogue computer both had 



t t 


Figure C. Transient response for Figure D. Transient response for 
step changes of desired value step changes of desired value 
(PI controller , Fig. 3) (FI controller , Fig. 4) 

the same settings as the ideal linear PI controller, and the same running 
time of the motor. The result proves that the arrangement of Figure 4 
is superior to that of Figure 3 as soon as the running time has an 
essential influence on the dynamic behaviour. 

(2) Contrary to the author’s opinion, there is no advantage in using 
relays. The results are the same if the relay (Figure 4 ) is replaced by a 
high gain amplifier with restricted output signal. The unfavourable 
influence of the sensitivity of the relay is not taken into consideration. 


The advantages appear only in the feedback circuit. The following 
arrangement (Figure E) is equal in dynamic behaviour to Figure 4. 

(3) As already pointed out in earlier publications 1 the controller 
Figure 4 is linear if the charge and discharge time constants of the 
feedback system are equal, if the restricted speed of the actuator is 
neglected. Unequal time constants result in an increasing or decreasing 
proportional band. Thus, in accordance with Dr. Pavlik, the quality 
of control decreases. 



Figure E 


(4) Dr. Pavlik points out that a closer approximation to the 
optimum control function will be achieved by adding a time lag to 
the controller or by increasing the running time of the motor. By this 
the damping ratio of the transient response increases. At the same 
time the integral of the absolute error increases too. In practice this 
effect is most undesirable. 

Reference 

Bottcher, W. Vergleich von Dreipunktreglern mit einem linearen 
kontinuierlichen PI-Regler. Regelungstechnik 10 (1962) 114-119 
and 210-213 


E. Pavlik, in reply 

(1) Many thanks to Mr. Bottcher for his discussion remarks. I am 
glad to see that he has essentially made the same observations when 
examining the systems shown in Figures 3 and 4, i.e. that the systems 
show a strongly different behaviour, the system of Figure 4 being 
unambiguously superior to that of Figure 3 . This is the more pleasant 
as Mr. Bottcher has taken the trouble to deal with the problems with 
a special modification of linear methods and not on the plane of the 
maximum principle. Generally valid determinations can hardly be 
made in this way, it is true, the systems being fundamentally non- 
mear in their character and can only be described by the exact 
methods of non-linear control theory. 

(2) The maximum principle requires the most exact utilization of 
the limited coordinate. Consequently, I cannot agree with the opinion 
of Mr. Bottcher that the application of relays brings no advantages. 
It is true that the control behaviour of the system shown in Figure 4 
is not substantially changed if the three-point switch is substituted by 
a continuous amplifier with a high gain degree and limited output 
signal, provided that the dynamic range of the amplifier is about 
identical with the response range (dead zone) of the relay. But in 
this case the continuous amplifier is substantially nothing but a relay. 

(4) This remark is the result of the translation error already 
mentioned in my reply to the contribution of Mr. Schreiner. Mr. 
Bottcher is also right, of course, in that an artificial time lag of the 
seivomechanism can only make the total control behaviour worse. 
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Nouvelle Procedure d’Optimalisation Statistique Fondee sur 

la Transformation 

F=(z-l)/(z+l) 

P. LEFEVRE 


Summary 

This paper deals with the optimization of sampled data systems sub¬ 
jected to several constraints and operating on noisy inputs; while signa 
and noise are supposed to be a random stationary, ergodic and sto¬ 
chastically independent time series, the optimization criterion is the 
R.M.S. error and saturating signals minimization. 

The^derivation of the optimum system is greatly simplified by using 
the V = (z —l)/(z + 1) transform, since the computation is done with 
the mathematical tools available for continuous systems. 

The derivation of the optimum digital compensator in the F-plane 
suggests a new optimization procedure as an extension of Newton’s 
theory. 

This method is applied to the following problem. Given (a) a zero 
order hold cascaded with a plant whose transference is H A (s) = 
1/0 2 (1 + r$) 3 ]; (b) the aforementioned input spectra, and (c) two 
acceleration constraints, find the optimum digital compensator to be 
inserted in the loop. 

The algebraic equations leading to the solution of this problem are 
thoroughly developed hereafter; numerical applications refer to 
the influence of the plant time constant r on the performance 
index; this is particularly important with digital systems, for it narrows 
the optimization zone; the greater this time constant is, the smaller 
the sampling period should be made. 

This paper also includes a table of the most usual F-transforms 
of transfer functions F(s). 

Sommaire 

Le present memoire traite de l’optimalisation des systemes echantillon- 
n6s soumis a plusieurs contraintes et operant en presence de bruit; 
le message et le bruit etant aleatoires stationnaires, ergodiques et in- 
dependants. Le critere retenu est la minimalisation des valeurs quadrati- 
ques moyennes de l’erreur et des grandeurs saturantes. 

La procedure d’optimalisation du systeme numerique se trouve tres 
simplifiee par Femploi de la transformee V 2 oil V = (z—l)/(z-j-l); 
tous lescalculs s’effectuant alors avec foutillage mathematique propre 
aux systemes analogiques. 

L’61aboration, dans le domaine (F), du compensateur arithmetique 
optimal suggere une nouvelle methode d’optimalisation qui constitue 
une extension de la theorie de Newton. 

Cette m<§thode a ete appliquee au probleme ci-apres: optimalisation, 
avec deux contraintes en acceleration, d’un compensateur numerique 
insere dans un systeme boucle, comprenant un circuit bloqueur d’ordre 
z&ro et un 616ment analogique: ff A (s) = l/[s 2 (l 4-rs) 2 ]. 

Sont indiquees toutes les relations litterales necessaires a la resolu¬ 
tion de ce probleme. Les applications numeriques portent sur l’in- 
fluence de la constante de temps r, a l’egard des performances opti- 
males. 

Celle-ci prend une importance particuliere en numerique, carelle res- 
treint l’intervalle d’optimalisation. Toute apparition ou dl6vation de 
constante de temps devra, le plus souvent, etre compens£e par un 
abaissement de la periode d’echantillonnage. 

Ce memoire presente egalement une table de transformees en F, 
les plus courantes, concernant les fonctions F(s). 


Zusammenfassung 

Dieser Beitrag befaBt sich mit der Optimierung von Abtastsystemen 
bei rauschformigem Eingangssignal, die verschiedenen Beschrankun- 
gen unterliegen. Da das Nutz- und das Rauschsignal als zufallsstatio- 
nare, ergodische und stochastisch unabhangige Zeitreihen angenom- 
men werden, gelten als Optimierkriterien der Fehler des Effektiv- 
wertes und ein Minimum an hohen Signalspitzen. Die Ermittlung des 
optimalen Systems wird durch Verwendung der Transformation F = 
(z— l)/(z+l) wesentlich erleichtert, da sich die Rechnungen mit den 
fur kontinuierliche Systeme verfiigbaren mathematischen Mitteln 
durchfuhren lassen. 

Die Suche nach dem optimalen digitalen Kompensator in der V- 
Ebene fiihrt zu einem neuen Optimierverfahren, das eine Erweiterung 
der Newton’schen Theorie darstellt. 

Die Methode wird auf das folgende Problem angewendet: 

a) ein Halteglied nullter Ordnung in Reihe mit einer Regelstrecke mit 
der Ubertragungsfunktion ff A (s) = 1/|> 2 (1 +T 5 -) 2 ], 

b) die oben genannten Eingangsspektren und 

c) zwei Beschrankungen fiir die Beschleunigung; 

gesucht wird der optimale in den Regelkreis einzufiihrende digitale 
Kompensator. 

Danach werden die algebraischen Gleichungen, die zur Losung 
dieses Problemes ftihren, ausfiihrlich abgeleitet; die numerische Be- 
rechnung dient vor allem der Untersuchung des Einflusses der Zeit- 
konstanten r der Regelstrecke auf die Giitezififer; dies ist bei digitalen 
Systemen besonders wichtig, weil der Optimierungsbereich dadurch 
eingeengt wird. Je groBer diese Zeitkonstante ist, um so kleiner soil im 
allgemeinen die Abtastperiode gewahlt werden. 

Der Beitrag enthalt daneben eine Tabelle der gebrauchlichsten 
F-Transformationen der Ubertragungsfunktionen F(s). 


Introduction 

L’optimalisation statistique du systeme echantillonne, sollicite 
par des signaux aleatoires stationnaires et ergodiques, a ete 
traitee jusqu’a ce jour uniquement a l’aide de la transformee 
en z 1-5 . Bien que Femploi de cette transformee soit tout a fait 
justifie du point de vue theorique, il n’en complique pas moins 
Fanalyse des cas concrets, pour les raisons enumerees ci-apres. 

(1) L’optimalisation en z engendre des operations de base 
tres differentes — comme le montre le Tableau 1 — de celles 
requises par l’optimisation des systemes continus, laquelle se 
developpe dans le domaine de Laplace (s). L’optimalisation du 
systeme echantillonne implique done la mise au point de pro- 
grammations speciales. 

(2) Comme nous Font revele de nombreuses applications 
traitees a l’aide d’un ordinateur 650 IBM, l’optimalisation en z 
devient rapidement imprecise et conduit meme parfois- a des 
resultats illusoires, des que se complique tant soit peu la structure 
de l’element analogique fixe a priori. Cette perte de precision 
est une consequence directe de Felevation du nombre de poles 
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et de zeros stables — lesquels se trouvent alors resserres les uns 
centre les autres a l’interieur du cercle unite — et de la necessite, 
d’evaluer, par la seule methode des residus de Cauchy, les 
integrates circulates du genre de celle figurant dans le 
Tableau 1. 

Cette situation nous a incite a entreprendre l’optimalisation 
du systeme echantillonne a l’aide de la transformee en v, deduite 
de la precedente par la relation 



L’inter6t d’une telle operation est de transformer le cercle 
unite du domaine (z) en l’axe imaginaire ja> du domaine (v) et 

Tableau 1. Principales operations effectuees au corns d’ me optimalisation 
_ statistique 

Optimalisation du systeme con- Optimalisation du systeme 

tinu dans le dom aine is) echantillonne dans le domaine (z) 

Zone de stabilite 


Decomposition d'une fonction 
paire par rapport a Vaxe ima¬ 
ginaire jeo 

ou / (s) renferme tous les poles 
et zeros stables de F (s) lesquels 
sont situes dans le demi-plan 
gauche. 

Evaluation de la valeur quadra- 
tique moyenne y 2 ( t ) en fonction 
de la densite spectrale ® y (s) 

* au-dessous 

^-^S-^ 0x(s)A(s) - 

A (-s) ■ B (s) -£(-s) • dr 

et puisque <I> x (s) est une fonc- 
tion paire 

—- 1 poo 

y 2 (t) = . | <px(jcd)A(jco) 

•Jitt o 

B(jco)\ 2 dco‘ 

avec 

(s) = (p x (s) • cp x (~s) 




Zone de stabilite 




f 




=~ 


Decomposition d'une fonction 
reciproque par rapport au cercle 
unite 

F(z) = /(z)/(z"1) 
ou /(z) renferme tous les poles 
et zeros stables de F (z) lesquels 
sont situes a l’interieur du cercle 
unite. 

Evaluation de la valeur quadra - 
tique moyenne d partir de la 
densite spectrale echantillonnee 

f au-dessous 

J' 2 (0 = A(z)-A(z~ 1 ) 


- ^ J A( Z ).A(z->) 
cercle unit6 
1>x(z)Z[B(s)-B(~s)]\ 


d etendre ainsi la zone de stabilite de ce dernier domaine a la 
totalite du demi-plan de gauche. L’analogie existant entre les 
zones de stabilite respectives des domaines (v) et (s) simplifie 
alors beaucoup la procedure d’optimalisation du systeme echan¬ 
tillonne en faisant beneficier celle-ci de tout l’outillage mathe- 
matique (Tables 7 - 8 , Programmations etc. ...) actuellement dis- 
ponible pour 1’optimalisation du systeme analogique. 

Ce memoire debutera par la presentation d’une table per- 
mettant de passer directement d’une fonction F(s) a sa trans¬ 
formee env: V [F(s)] ou F(v) — puis exposera, dans ses grandes 
lignes, la methode d’optimalisation statistique en ( v ) applicable 
au systeme echantillonne, en presence de bruit et de plusieurs 
contraintes. II s’achevera sur un exemple d’application, exami¬ 
nant l’influence exercee par les constantes de temps de 1’element 
impose, a l’egard du choix de la periode de prelevement T. 


Generalites sur la transformee en V 

Definition et mode d'utilisation 

Soit une fonction fit) definie pour tout t et possedant une 
transformee de Laplace 

r<»=| / (t)e _sf dt 

la transformee en z bilaterale est definie par 

^0)= f / (nT)z~" (2) 

n— — oo 

et la transformee en v qui lui est associee s’ecrit: 

f w 4 ['(>)U- i /omfeT ® 

1 v «=-oo \ J +7 

La table, reportee a 1’annexe /, indique les transformees en v 
des fonctions F(s) les plus couramment rencontrees dans un 
probleme d’optimalisation. 

En ce qui concerne son mode d’utilisation, la transformee 
en v obeit strictement aux memes regies que la transformee en z. 

Expression de la valeur quadratique moyenne dans la domaine v 

La methode de Sklansky 6 , appliquee au systeme echantillonne 
du Tableau /, permet d’ecrire la valeur quadratique moyenne de 
la sortie y(f) sous la forme suivante: 

unit6 

la transposition dans le domaine ( v ) s’effectue aisement. On 
trouve: 

EW = j4 - P A(v)A(-v)® x (v)VlB(s)B(-s)-]Z^ 

J J -Joo 1—V 

Les fonctions <P x (v) et V [B(s) B (- *)], etant paires, leur de¬ 
composition par rapport k l’axe imaginaire jw s’effectue comme 
celle de la fonction F(s) du Tableau I: 


{$* (v) = (p x (v) <p x (-v) 

1 V l B 0) B (-s)] = b (v) b (- v) 
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Dans ces conditions, la valeur quadratique moyenne (5) 
s’exprime comme suit: 


y\ty- 


2 

Tn 


j : 


\A(jco) (p x {j(p)b ( jco ) 


dco 


1+7 co 

revetant ainsi la meme forme que pour le systeme continu. 


(7) 


Procedure d’optimalisation dans le domaine ( V ) 

La methode, exposee dans ses grandes lignes ci-dessous, cons- 
titueune extension aux systemes echantillonnes, des theories d’op¬ 
timalisation de Wiener et Newton 7 elaborees pour les systemes 
continus. Elle s’apparente done a la procedure utilisee par divers 
auteurs, notamment par Chang 2 , Tou 3 , Thellier 5 , au cours de 
leurs travaux dans le domaine (. z ). Elle va etre precisee a l’aide 
du systeme boucle de la Figure 1 . Celui-ci est suppose recevoir 




Jx-X. 


D(e sT )U->- 


H 0 (s)| 


r--iM i (s)_J-- yi 


H f (s) 


Figure 1. Schema de Vasservissement considere 


deux signaux aleatoires stationnaires, ergodiques et non corre- 
les: le message x a (J) et le bruit x h (t ), dont les densites spectrales 
respectives 0 a (s) et <P h (s) sont connues. 

Ce systeme renferme, d’une part, un circuit bloqueur d’ordre 
zero H 0 (s) et des elements analogiques, lineaires et invariants 
dans le temps, dont les transmittances H f (s) et Mt(s) sont 
lixees a priori , et, d’autre part, un compensates arithmetique, 
de transmittance D(c sT ) inconnue, sur lequel repose l’optimali- 
sation. 

Le probleme consiste a elaborer ce compensateur de faqon 
a rendre minimales les valeurs quadratiques moyennes de 1’er- 
reur: e(s) = x a (s) — x JS (s) et de certaines grandeurs satu¬ 
rates yi(s). 

Cela revient a prendre, comme critere d’optimalisation, la 
minimalisation de la quantite suivante: 




dy 


+ D, (v) D s (-v) [O a (v) + d>„ (»)] ■V[0(s)Q(-s)-]} 1 ^ 


yf(t)= 


nTj 


DMD s (-v)(<S> a (v) + Q b (v)) 


dv 


• V [M, (s) M ; ( - s) H 0 (s) H 0 (- 5 )] 

l—o 

et la quantite Q n revet alors la forme suivante: 


( 10 ) 


( 11 ) 


Qn = X‘(t) + 


1 P" 

TnJJ-jo 


dv 


l-D s (v)F(v)-D s (-v)F(-v)+D s (v)D s (-v)C( o)]^ 

( 12 ) 

avec: 

(F(-o)^F[e(-s)<5 a (s)] 

C(»)4k[2W0(-»)] 


(13) 


+ I [M ; (s) M t (—s) H 0 (s) H 0 (- s)] 

•(® a (»)+«*(»)) 


Le compensateur optimal, qui sera designe par, D s0 (v), 
devant rendre minimale la quantite Q m se deduit de la relation 
(12) au moyen du calcul des variations. Cela consiste a poser 
D s (v) = D s0 (v) + q 3(v) oil, q, est un parametre reel et, d(v), 
une fonction appartenant au meme ensemble que la fonction 
D s0 (v ) et a satisfaire la condition 




Q eA= 


e = o 


Q n =£ 2 (F)+ £ kyf(t) (8) 

i = 1 

ou, X h designe le parametre de Lagrange associe a la contrainte 
relative au signal saturant y t . 


La minimalisation de Q n est obtenue pour: 



5{-v) 




-F(-v) + D s0 (v)C(v) 
l-v 2 


du = 0 


x b _ 
iSii_x. 


-.[Mijs)!- y 


D s (e sT )P - i H 0 ts) H — 


Figure 2. Representation fonctionnelle de Voptimaiisation statistique 

On passe aisement du systeme boucle initial au montage en 
serie equivalent de la Figure 2, en posant: 


D,(v)=- 


D(v) 


-avec Q(s)±H 0 ( S )HAs) (9) 


Les valeurs quadratiques moyennes s’expriment comme suit: 


e 2 (0 = * 2(0 


+ 


1 

nTj 


{-D s (v)V[Q (s)0„(s)] 


Cette equation implique l’analyticite, dans le demi-plan 
gauche, de la quantite suivante: 


F(-v) 


+ 


D,oWC + W 


en posant; 


(1 + u) C~(v) (l + o) 
C (v) — C + (v) C~(v), 


(14) 


C + (v) contenant tous les poles et zeros stables de C(v). 

II en resulte 1’expression du compensateur equivalent optimal: 


D s0 00 = 


(l+u) 


F(-v) 


C» L(l+»)C-(i>)J + 


F(-v) 


(15) 


L(i+®)c-(»)j 
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designant la partie de la decomposition de la fonction rationnelle 

F(~v) 

(1 + v)C~(v) 

en elements simples qui contient tous les poles stables. 

L’expression du compensateur optimal D 0 (v), a inserer dans 
le systeme boucle initial, decoule de (9) et (15). 

Exemple duplication de Poptimalisation en V 
Presentation du probleme 

On se propose d’optimiser le systeme schematise a la Figure 3, 
dans lequel la transmittance imposee renferme une double cons- 
tante de temps, t, montee en serie avec une double integration: 


T _ T _ c _, E 

-V*- D (v) —^ T _jjzxrij_m 

i L_i s (uts)2 S 2 


Figure 3. Structure du systeme a optimaliser 

as; 

les signaux d’entree font 1’objet des donnees suivantes: 

— Ia densite spectrale du message x a est definie par: 

*•<'>%£?) ( 17 ) 

— le bruit x b est suppose blanc dans Fintervalle 

( n ^ 7 z\ 

V 

de niveau k 2 et nul ailleurs. 

Le probleme consistera d’abord a rechercher le compensa¬ 
teur arithmetique optimal D 0 (v) qui minimalise la quantite 


u=bT (20) 

et des coefficients A i} 0 h L, h M u G 7 et r i explicites a l’Appen- 
dice II. 

Calcul des transformees en v — 




v ( 1 — coth 2 — v : 


en posant: 


A (v) — 1 A 2 v 2 + A^v^ A^ 


Calcul de C(v). D’apres (13), on a 


en posant 




L -s°(1-t^s 


+a c t +a e v 


-e“ Ts )(l-e Ts )l 

^'(l -^ 2 ) 2 J 

'(1—e" rs )(l-e rs )l 
. —s 2 (l— t 2 s 2 ) 2 J 


Cette derniere fonction depend done a la fois de la structure 
de 1 element analogique imposd et des contraintes retenues. En 
revanche, elle reste independante des signaux d’entrde. Les cal- 
culs procurent: 


22~£ 2 (0+(18) 

dans laquelle interviennent les parametres de Lagrange, l c et X E , 
amsi que les valeurs quadratiques moyennes de Ferreur de 
Pacceleration de sortie y E et de Pacceleration commandee yc. 

II s agira ensuite d’evaluer les performances realisees par 
1 asservissement optimal done de chiffrer, pour tout ensemble 
(Ac, Ae) fixe, les grandeurs suivantes: 

(a) Erreur efficace: 

(b) Acceleration de sortie efficace: 

a ys~{yE (0) a 

(c) Acceleration commandee efficace: 

*ro±(m* 

Calculs d?optimalisation 

, Les diverses expressions, engendrees par Papplication de la 
theorie que nous avons developpee precedemment, vont etre 
precisees a Paide des deux parametres: 


v 1 — coth 2 — v‘ 


$ (v) = 1 + <P 2 v 2 + + <P 6 v 6 + # 8 Z> 8 

Les relations (21), (23) et (25) conduisent a: 


a 2 T 10 
= (16) 2 w 2 


A(v)0(v) 


v \ l-coth ; 


. u 7 

T v \ 


1 — coth 2 —v 


Calcul de F(— v). 


F(- v )=-a 2 V 


s 1 (l — rs) 2 (b 2 — s 2 ) 


le developpement d’une telle transformee ne peut etre indique 
ici, en raison du manque de place. 

Expression des compensateurs arithmetiques 
Compensateur serie equivalent optimal. Le developpement 
de (28) joint a la relation (27) permet d’ecrire la relation (15) 
sous la forme suivante: 


D s 0 (v) = ~, 


® 2 (l + ®)(l + coth—o) L(») 
a ( v ) (p ( v) 
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en posant: L 0) £ 1 + L t v + L 2 i; 2 


(30) Acceleration de sortie: 


et en designant respectivement par oc(v) et <p(v) les polynomes 
resultant de la decomposition, par rapport a Paxe imaginaire, 
des fonctions A(v) et &(v): 


ot(v)a( — v) = A(v) 
cp (v) cp{ — v) = <P (v) 


Ils se developpent comme suit: 


yl( t) 


avec 


(3i) r E (v)=- 


(i+») 


_2flT 

Ttu 2 


X 


\r E (jco)\ 2 dco 


1 + coth 


coth 2 ~-l 


1 + coth ^-v](p(v) 


( 37 ) 

•L(tO 


a(u) = l + a 1 v + x 2 v 2 + a 3 v 3 
<j)(v)=l + (p 1 v + cp 2 v 2 + q> 3 v 3 + cp 4 v 4 


(32) 


Compensates optimal D 0 (v). II se deduit de l’expression de 
D s0 (v) et de la relation (9). Cette derniere renferme la trans- 
formee Q(v), definie par: 


Q(v)=v f 


— e 


-sT 


(1 -t-TS) 2 


_ T 2 (l — v)MJp) 
4v 2 (l + coth-^-t> 


avec: 


M(v) = l + M 1 v+M 2 v 2 + M 3 v 3 
On obtient ainsi l’expression suivante: 


(33) 


(34) 


. (l+i>)( 1+coth —u ) ■ L(v) 

Do(v)=~^ - V 


G 0 T 2 


g(v) 


(35) 


ou 


g(v) = l+g 1 v + g 2 V 2 + g 3 v 3 + g 4 v 4 + g 5 v s 


(36) 

avec gi = GJG 0 , les coefficients Gi etant definis a PAppendice II. 

Expression des valeurs quadratiques moyennes — Avec les 
fonctions introduites plus haut, les valeurs quadratiques prennent 
les formes simples suivantes: 


Erreur £ 2 (0=e a 2 OW(0 

Composante message: 


avec 


£.00 = 


_ „2t-t5>o2 f* oo 

:!(t)= ~8^j 0 l E Ml 2dtu 

r(v)g(v ) 


1 + coth — v]ot(v)(p ( v ) 


et 


r^—l + r^ + r ^ 2 
Composante bruit: 


A cceleration commandee: 

VcO) 


avec 


2 a 2 T 


Till 


| r c (jffl) | 2 doj 


(38) 


1+coth —v 1 L(v) 

r c (r)A.' 2 


1 + coth — v ) cp (v) 


Applications numeriques: influence de la constante de temps t 
sur le choix de la periode de prelevement 

Cette etude a necessity la resolution de 40 equations de 6e 
ou 8e degre et le calcul de 200 integrates environ. 

Donnees numeriques — Les resultats presentes ci-apres 
concernent les donnees suivantes: a 2 = 9 (m 2 /sec 5 ); A: 2 = 14 
(m 2 /rad/sec); b = 2/45 (sec -1 ) — 0. L’optimalisation s’effectue 

done avec la seule contrainte en acceleration commandee: X c ^ 0. 
Les parametres de l’etude sont constitues par la periode T et la 
constante de temps x de l’element analogique. 

Presentation des resultats 

Courbes de performances et point d"arret. On designe par 
courbe de performances, le graphique de la variation de Perreur 
efficace o e en fonction de la valeur efficace de l’acceleration 
de sortie o Vjs , relatif a une valeur fixee des parametres T et r. 

La Figure 4 (a) montre deux courbes de ce type relatives a 
r = 0,1 sec et a 2 valeurs de T. Le cas T = 0 represente le cas 
analogique traite selon la methode de Newton. On constate que 
la courbe T = 0 est asymptote a une certaine valeur ( o e ) L 
correspondant a Poptimisation de Wiener (X E = 0), tandis que 
la courbe T = 0,4 sec, toujours situee au-dessus de la courbe 
precedente, presente un point d’arret d’abscisse (oy^M* L’exis- 
tence d’un tel point s’explique mathematiquement du fait que le 
degre de la fonction 0 (v) reste invariant lorsque s’annulent les 
parametres Xe et X c [voir relation (37) et Annexe II]. 

On sait deja 5 que l’abscisse (oy^M du point d’arret decroit, 
lorsque T augmente, et qu’elle constitue ainsi un element fonda- 
mental pour le choix de cette periode. 

La Figure 4 ( b) indique les variations de Pacceleration com¬ 
mandee, cry c , le long des courbes precitees. Le filtrage de cette 
acceleration est plus energique en digital qu’en analogique. 


avec 


% (0 = 


2/c 2 

7lT 


\Eb(joS)\ 2 &co 


E b (v) 


^ (l+t?)L(i))M(u) 

<x(v)(p(v) 


Influence de la constante de temps r. Les points d’arret du 
systeme digital ont 6te calcules pour de nombreuses valeurs de T 
et de x. Ils font l’objet de la Figure 5 sur laquelle ont ete traces, 
en traits interrompus, les lieux de ces points a r constant, et, en 
traits pleins, les courbes de performances du systeme analogique 
concernant les mdmes valeurs de r. On constate que la constante 
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Figure 4. Courbes relatives a r — 0,1 sec. (a) courbes de performances; (b) variations de Vacceleration commandee 



Figure 5. Lieux des points d’arret des courbes de performances a x 

constant 


de temps n’intervient pratiquement pas dans l’accroissement de 
l’erreur consecutif au passage de Panalogique au digital; celui-ci 
etant essentiellement imputable a la periode T. L’evaluation de 
cet accroissement, a la hauteur du point d’arret, procure: 

T (sec) 0,2 0,4 0,8 

(en %) 10 28 70 

En revanche, la constante de temps joue un role tres important 
a 1 egard de 1 intervalle d optimalisation, dont la borne superieure 
est constitute par l’abscisse du point d’arret. Comme le montre 
la Figure 6, un accroissement de r se traduit toujours par une 
reduction de cette abscisse, d’autant plus accentuee que la 
periode T est plus faible. Le maintien de l’intervalle d’optimalisa- 
tion sur une longueur donnee exige que tout accroissement de la 
constante de temps soit compense par une reduction de la pe¬ 
riode T. Par exemple, le maintien de (o Ye ) m sur 240 (m sec -2 ) 



0,1 0,2 0,4 0,8 

T(sec) 


Figure 6. Variations de Vabscisse du point d’arret, avec r 



Figure 7. Acceleration commandee pour les points d’arret de la Figure 5 
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Figure 8. Variations de a ye /a yE deduites de la Figure 7 


implique, lorsque r s’eleve de 0.15 a 0,37 sec, que la periode T 
soil reduite de 0,2 a 0,1 sec. Par ailleurs, Facceleration comman- 
dee efficace a ete calculee en chacun des points d'arret de la 
Figure 5. Le graphique de la Figure 7 indique les variations de 
oy c en fonction de (ay E ) M . L’exploitation de ce graphique conduit 
au trace de la courbe unique de la Figure 8 , d’apres laquelle, le 
filtrage de ye par rapport a y c ne depend que du quotient r/T 
et croit rapidement avec ce dernier. Un tel resultat pouvait, 
dans une certaine mesure, etre predit a Faide des relations (37) 
et (38). 

Conclusion 

Les constantes de temps contenues dans l’element impose 
influent beaucoup plus sur Foptimalisation digitale que sur 
Foptimalisation analogique. Leur action sur celle-la est particu- 
lierement nefaste puisque restreignant Fintervalle d’optimalisa- 
tion. Toute apparition ou elevation de constante de temps doit, le 
plus souvent, etre compensee par un abaissement de la periode 
de prelevement. 

& auteur adresse ses remerciements a la Direction des Etudes 
et Fabrications d'Armement grace d laquelle cette recherche a pu 
etre entreprise et menee d bonne fin. 


Appendice I 


Tableau des Transformees en v 


Fonctions S 

Transformees en v 

Fonctions S 

Transformees en v 


coth 


ST 


1_ 

~S 


1 + v 
2v 


^ST 


S + ci 


1 +v 
i —v 

(1 + 1)) ^COthy+1 


2( 1+coth —v 


1 

S 2 

1 

S 5 


T (1 —v 2 ) 
2 2 v 2 


T 2 ( l-v 2 ) 
2 3 u 3 


(s + ci y 


(S + a)' 


(S + ci ) 4 


T(l-u 2 )(coth 2 y-l 


2 2 -( 1 +coth~u 


T 2 (.l -y 2 )(coth 2 -^—1 j^coth-y + u 


,2 * 


T (l-v )[ coth 


2 4 ■ 1 + coth —v 


+ yl) + U 2 + (l-ylcoth 2 X 


1 

s 4 

1 

S 5 


1 

S 7 


1 

s 8 


T 3 (l-v 2 ) 


2 4 v 4 ' 


r 4 (i-u 2 ) 


2 v 


l ~~T v 


1 2 2 
1_ T 


x v 

1 

T 5 (1 — v 2 )" 

1 2,2 4- 

2 V ) 

S 5 

2 6 w 6 

i-„ + B » j 


T 6 (l-v 2 )[ i 4 2 17 7] 

+ 45 y J 


^d-or, 

"w~"L 


5 2 11 4 17 6 

+ 15 y -Bis 1 ’ 
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Functions S 


Transformees en v 


Appendlce I—continued 
Fonctions S 


Transformees en v 


j ! (1 — y 2 ) coth — 

S 2 -a 2 ~2a X J T - 

(l-COth 2 yZ , 2 


Nota On a pose x=aT 


T s ( l-o 2 )^ „ 2 . 6 4 62 fi 

“ 2|V 1_2 " + T U “ 315 ° 


^ (1 v ) < 7 2 . 16 4 88 62 

+ T' -189- +2835 


Appendice II—Determination des coefficients relatifs a I’exemple 
d’application de I’optimalisation en V 

Coefficients A% 


Coefficients 


# 2 =-l-2coth 2 ^+-l 


4 2 u 4 

■T-coth 2 —+-j 


a- 1 4 4. 8 rAjh u , a/l l\ ,.2 « 16 k 2 u 2 

^-T-? + P C0th T +4 (T~^J C0th T+^f 5 


# *-n + 2 l 1 -?J 00th T 


, it 4 x 8 48 16 . 

+cotk T~J F + ? + F (x * +w 


*•" “?- 4 (n-3? + ?) C0,hI f+fe- 1 ) Mh ‘- 


Coefficients Mi 


M 1 = ll coth 


x_ 2 
2 x 


^112 , x 

H—5-coth — 
x 5 2 


M 2 = coth — 


x 6 


2 x 


M 3 =— 1-3 (coth 


2 x 


-1-3 coth 2 y+y coth y j - 2 A c coth 2 


2 x 2 


2 x 


Coefficients Gi 


*8 = co th2 y{-4—^cothf +2 (l—^)coth 2 | 


+^p ^(-l + 3coth 2 y-ycothy) + Ac coth 2 y 


Coefficients Li 


4 x 

L l =«! + <?!+- 2 coth— 

X 2 


L 2 =(l 1 —coth y j coth y 


64 ( th y) (^1 + cothythyj 

“‘( 1+ t) “(•Hr) ’’(“’t) 


G 0 = ia 2 + tx t tp t + cp 2 +1 - (L 2 + LjAfi + M 2 ) 

G 1 =a 3 +a 2 (p 1 +tx 1 (p 2 + <p 3 +(a 1 + (p 1 ) 

~(L 2 M 1 + L 1 M 2 + M 3 ) 

G 2 =cc 3 cp 1 +x 2 q> 2 +<x 1 (p 3 +(p 4 + (L 2 + L 1 M 1 +M 2 ) 

— (L 2 M 2 + L 1 M 3 ) 

G 3 = cc 3 cp 2 + cc 2 ( p 3 +x 1 (p 4 + {L 2 M 1 + L 1 M 2 +M 3 )~ L 2 M 3 
G 4 = ix 3 cp 3 +cc 2 cp 4 + (L 2 M 2 + LjMj) 

Os = x 3 cp 4 + L 2 M 3 


Coefficients n 

r 2 = (R 2 )* 


r i = (2 r 2 —R 1 ) i 


K 2 = cothy[(i—-^)co,h-| 

14 ^ o W 

^i— ""•y + -2~coth 2 — 


u 8 

T + i?_ 


678 


NOUVELLE PROCEDURE D’OPTIMALISATION STATISTIQUE FONDJ&E SUR LA TRANSFORMATION 


y = (z — 1) / (Z + I) 


References 

1 Ragazzini, J. G. and Franklin, C. Sampled-Data Control Sys¬ 

tems. 1958. New York; McGraw-Hill 

2 Chang, S. S. L. Statistical design theory for digital-controlled 

continuous systems. Trans. Amer. Inst, elect. Engrs II (1958), 77 

3 Tou, J. T. Digital and Sampled Data Control Systems. 1959. 

New York; McGraw-Hill 

Bergen, A. R. On the statistical design of linear random sampling 
systems. Automatic and Remote Control, p. 430. 1960. London; 
Butterworths 

Thellier, P. L. Optimisation et auto-optimisation des Systemes 


de commande a donnees echantillonnees en presence de satura¬ 
tions. Mem. Artillerie Franc. T 35 (1961), 3e fasc. 

6 Sklansky, J. On closed-form expansion for mean squares in 

discrete-continuous systems. Inst. Radio Engrs., Trans. Automat. 
Contr. PGAC 4 (1958), 24 

7 Newton, G. C, Gould, L. A. and Kaiser, J. F. Analytical De¬ 

sign of Linear Feedback Controls. 1957. New York; Wiley 

8 James, H. M., Nichols, N. B. and Phillips, R. S. Theory of Servo 

Mechanisms. 1947 

9 Lefevre, P. L’optimisation Statistique du Guidage par alignement 

d’un engin autopropulse en presence de bruit. Congris A. G.A.R.D. 
sur la Stability et le Controle. Bruxelles Avril 1961 


DISCUSSION 


R. E. King, Electrical Engineering Dept., Queen's University, Belfast, 
N. Ireland 

Closed-form expressions for determining the mean squared values of 
signals in discrete-continuous systems have been obtained by Sklansky 1 
and Mori 2 . With reference to Figure A where A* x {qsT) i s the ‘discrete 


u(t) / u*(t) 

T 

Auxte S ^) 

xlt) | 

I G xv (s) 


/ v ; 

v(t) 





- i 




.Discrete 



Continuous_ 



1 element ~ ! 


element 



Figure A 


transfer transform’ in the system and G xv (s) is the continuous part, 
the value of the mean squared output, for example, can be expressed 
in either of two forms: 


dz 


» 2 = 2 TjJ c ' n(2) l^ x (z)\ 2 Z[\G xv (s)\ 2 ] ^ 

due to Sklansky, and 


^=2 


\G* v (z,m)\ 2 dm — 
LJ o z. 


due to Mori, using modified 2 transforms, where 


( 1 ) 

( 2 ) 


Tables of symmetrical modulus squared z transforms | T* (z)| 2 for 
the most commonly found types of transforms G^Cs) have been 
derived 4 for use in the analysis and optimization of amplitude de¬ 
pendent discrete-continuous systems. An example is 



GxJs) ~s 2 ( S + a) 

for which 


ir*(z)| 2 = 

Z [\G xd (s) 1 2 ] 

II 

( c o c i 4“ c-i) z +2(c 0 — c 2 ) z 4- (c 0 — Ci 4- c 2 )| 

96 a 5 

(z-l) 2 (z-e~“ r ) | 


where 2 

c o~ 2 £q + 2 Qi + q 2 


c 2 = 2 £ 0 — 2 Q 1 + Q 2 

c i = —12 q 0 + 2 q 2 +2c 0 c 2 
Q 0 = 6 sinh aT-6 aT-(aT) 3 
Ql = 12 aT(\+ cosh aT) 
-2(aT) 3 (2- cosh aT) 
-24 sinh a T 


X K 0) = 0) 0 ~ *) = I 0) 1 2 


£2 = 36 sinh a T 


is the input pulse spectral density. 

In general, for physically realizable stable systems the z transform 


z[K?*„(s)| 2 ] 

can be expressed in a symmetrical modulus squared form |T*(z)| 2 
Thus Sklansky’s integral becomes 


a rational fraction of complementary conjugate polynomials in z, 

J (z) == — | Pn — 1 ~hPn- 2 Z ~h---jPo | dz 

2n jJc | + 2 


(3) 

i.e. 

(4) 


By using the bilinear transformation 


1 + 2 
Z— 1 — 2 

these can be expressed in the standard form given elsewhere 3 . 


+ 2(ar) 3 (4coshaT-l) 

—12 aT(l + 2 cosh aT) 

An example of the application of the modulus squared z transform 
to optimum system synthesis of a fixed configuration problem is 
shown in Figure B. Here 4 contours of equal mean squared error ratio 
(<? 2 /w 2 ) of a particular sampled data system are shown for a range of 
values of the two free system parameters a> 0 and f. The optimum point 
(a>*,£*) gives the minimum possible mean squared error of the 
particular system configuration when the sampling period and input 
spectrum are fixed. 

However, in practical systems where non-linearity is inherent there 
exist certain fundamental system constraints, as for example the 
torque capacity of the system. Thus the movement of the parameters 
in the co 0 —f plane must be constrained if a realistic optimum is to be 
achieved. 

In order, therefore, to constrain the mean square value of a system 
variable, in an attempt to avoid distortion, further sets of loci of the 
mean squared value of the signals to be constrained may be super¬ 
imposed on the mean squared error ratio loci of Figure B. Where the 
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system is subject to one constraint, co 2 ^ co^ then a ‘restricted’ 
optimum must be found on the locus. In Figure B the true optimum 
is as shown at (wJ,C*) and the ‘restricted’ optimum at (co* oR , £#). 
This in effect is a graphical Lagrangian technique 5 and is of particular 
use where analytical methods are exceedingly complex. 
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P. Lefevre, in reply 

The use of the variable v (or X) as suggested by Dr. King is exactly 
what we did a few years ago and now deem superfluous for the follow¬ 
ing two reasons: (1) For very complex sampled-data systems, the 
change of variable comes too late to be really efficient from the 
viewpoint of computing accuracy. With respect to the v domain 
optimization, it involves an appreciable number of additional opera¬ 
tions for passing from eqn (3) to eqn (4), as also for introducing the 
variable v. 

Furthermore, it does not make it possible to avail oneself of the 
simplicity of the v transforms with respect to z transforms for com¬ 
plex systems, as shown for example by the respective transforms of 
1/s 9 below: 

z (^) = lb 7 + 1 + 247 z( 1 +z 5 ) 

+4-293 z 2 (l + z 3 ) 

+ 15-619 z 3 (l + z) 

1 - 2 f 2 + t f 4 -S^ 6 _ 

(2) For reasonably simple systems the change of variable proposed 
is useless, because of the recent publications of Professor Jury’s table 
of integrals, which permit a direct determination of integrals of 
type (4). 


A detailed exposition of the v domain method of optimization will 
be found in Vol. 3 (in preparation) of the series ‘Progress in Automatic 
Control Engineering’ (Heywood, London), edited by Professors 
Macmillan, Naslin and Higgins. 

B. M. Brown, Royal Naval College , Greenwich , London , S.E. 10 

Dr. Lefevre has produced an interesting solution of the problem 
of optimization of a sampled-data linear system. He claims two ad¬ 
vantages over the more direct method making use of the z transform 
and I would like to discuss these in turn. 

The first advantage claimed is that by making the bilinear trans¬ 
formation v — (z — l)/(z + 1), the method used in the analogous 
problem for a continuous system becomes applicable. This is of 
course true, but the corresponding method for a discrete system is 
very similar analytically and no more difficult to apply. Indeed if it is 
desired to have a common method to cover the two types of system 
a case could be made out for taking the discrete system as basic, for 
in this case one is dealing with Fourier series rather than Fourier 
integrals, and integrals round a finite closed contour rather than along 
an infinite line. Results and methods for discrete systems could then 
be applied to continuous systems by the simple device of sampling. 
Provided that the various input functions of continuous time are 
stationary statistically the mean squared error found at the sampling 
instants will be equal to that at any other instant. 

The bilinear transformation is frequently quoted as a device for 
reducing discussion of the stability of a discrete system to that of a 
continuous system. In fact it turns out to be rather disappointing for 
this purpose. Unless the degree of the characteristic equation is low 
the arithmetic or algebra involved in carrying out the transformation 
is not insignificant. As Professor Jury has reminded us earlier in this 
Congress, there are available direct methods for discrete systems 
analogous to and no more difficult to apply than those of Routh and 
Hurwitz for continuous systems. The bilinear transformation is not 
even particularly helpful in deriving one set of conditions from the 
other. 

In general, therefore, it would appear that, other considerations 
being equal, it is better to use the particular method appropriate to the 
particular type of system. However, Lefevre reports that he has had 
arithmetical difficulties when the various integrals are expressed in 
terms of z due to concentration of the poles of the integrands inside 
the unit circle. These could well justify the transformation to v I 
wonder, however, whether the difficulties could be avoided by evaluat¬ 
ing the integrals in terms of z by other methods, such as the application 
of Cauchy’s theorem to the poles outside the unit circle or the use of 
Parseval’s theorem. Alternatively, a table can easily be produced 
which gives integrals of this type directly in terms of the coefficients 


1 \ T 8 (1 — V 2 ) 

7r^~v~~ 
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TRANSFORMATION V - (Z— 1) / (Z + 1) 


of the numerator and of the partially factorized denominator of the 
integrand. Such a table would be analogous to that which has been 
calculated for continuous systems. 

P. Lefevre, in reply 

The two following points in Professor Brown’s comments seem to 
apply directly to statistical optimization. 

(1) It is suggested that the statistical investigation of continuous 
systems could be carried out by the method developed for sampled-data 
systems, by providing the system with a fictitious sampler. This pro¬ 
posal seems to be carrying a bit too far the desire for a standardization 
of the method! It does not seem to agree, either, with the arguments 
expounded in the oral presentation of the paper. 

(2) The determination of the R.M.S. values by the method of 
residues or by means of a table of integrals is quite interesting, as 
mentioned before, for simple systems, but proves impractical for very 
complex structures. 

Ya. Z. Tsypkin, Institute of Automation and Telemechanics , Kola - 
chevskaya 15-A, Moscow 1-53 , U.S.S.R. 

In the report, the author proposes to use the well-known transformation 
Z “ I” for lhe stat isti ca l optimization of impulse systems, which 
was used for the same purpose in other papers 1 ’ 2 . The application of 
this transformation changes the problem of statistical optimization 
of impulse systems to the statistical optimization of analogue systems. 
But using digital computers, we should again be sampling, which 
means that auxiliary errors will be introduced. Thus, the method of 
solution proposed uses unnecessary transformation of the initial 
equations of the impulse systems with sampling following; therefore 
it is easier to use the z transform (or D transformation). As for the 


evaluation of loop integrals which the author considers problematical, 
I hope that all difficulties can be avoided by using the method of 
polynomial equations developed by Volgin 3 . 

Examples of optimization of impulse systems which can easily be 
handled on digital computers are given in Volgin’s book 3 and in my 
own book 4 . The latter way seems to be preferable, both with respect 
to the number of operations and the accuracy that can be obtained. 
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P. Lefevre, in reply 

1 wish to thank Professor Tsypkin for his interesting comments. I find 
it rather difficult to formulate my answer because I was not up to now 
aware of Volgin’s method which does not seem to have been available 
at the time I carried out my work. My only information concerning 
the method is that it requires the transformation of the various 
functions involved in the form of polynomials. In this respect, it would 
seem to be related to the methods mentioned by Professor Brown and 
Dr. King. Finally I should like to ask Professor Tsypkin whether 
Volgin’s method has actually been applied to systems of the same 
order of complexity as those we mentioned. 
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Summary 

Certain processes which require to be controlled by means of feedback 
loops have small perturbation static or dynamic characteristics whose 
parameters are functions of the unperturbed output level. The form 
of the response of such systems to large disturbances may be a signifi¬ 
cant design requirement. The paper describes the first results of a study 
of the response of controlled systems of this type, and their relation 
to characteristics which can be deduced from an extension of the 
small-perturbation roots-locus method of analysis for such processes. 

Results are described for the proportional control of processes 
whose small-perturbation transfer functions consist of an output- 
dependent gain and either a simple fixed pole or a complex fixed 
pole-pair. 

For the first controlled process the step response for large disturb¬ 
ances is similar to that for a first-order linear system. The time- 
constants of nearly equivalent linear systems are therefore derived in 
terms of the pole-positions of the small-perturbation roots-loci. For 
the second controlled process the step response for large disturbances 
may be stable or unstable. Criteria for stability and the form of stable 
response are investigated by means of Liapunov's Direct Method , 
a phase-plane analysis , an extension of the roots-locus concept to that 
of a small-perturbation ‘roots-surface\ and an approximate method of 
analysis . The ‘roots-surface’ concept appears to be a promising 
method of analysis of the response of controlled processes of this class, 
and the investigation is being extended. 

Sommaire 

Certains processus exigeant une commande par boucles a reaction ont 
des caracteristiques statique et dynamique sous faible perturbation 
dont les parametres sont des fonctions du niveau de sortie non 
perturbe. La forme de la reponse de tels systemes a de fortes per¬ 
turbations peut constituer une condition importante de la conception. 
Le rapport decrit les premiers resultats d’une etude sur la reponse de 
systemes commandes de ce type, et leur relation avec des caracteristi¬ 
ques pouvant se deduire d’une methode d’analyse du lieu des racines 
sous faible perturbation. 

On decrit des resultats concernant le reglage proportionnel de 
2 processus dont les fonctions de transfert, sous faible perturbation, 
consistent en un gain dependant du niveau de sortie et un pole fixe 
simple, ou un couple de poles fixes complexes. 

Pour le premier processus regie, la reponse a l’echelon unite 
pour de fortes perturbations est semblable k celle de systemes lineaires 
du premier ordre. Les constantes de temps de systemes lineaires a peu 
pres equivalents sont done deduites en fonction de la position des 
poles du lieu des racines sous perturbation faible. 

En ce qui concerne le second processus regie, la reponse a 
Pechelon unite pour une forte perturbation peut etre stable ou instable. 
On examine alors les criteres de stabilite et la forme des reponses 
stables au moyen de la methode directe de Liapunov, une analyse 
du plan de phase, une extension du concept du lieu des racines a celui 
de «surface des racines » sous perturbation faible, et une methode 
d’analyse approchee. Le concept de «surface des racines » parait etre 
une methode prometteuse d’analyse de reponse de processus regies 
de ce type, et son etude se poursuit. 

Zusammenfassung 

Gewisse geregelte Systeme besitzen bei kleinen Storungen ein stati- 
sches und dynamisches Yerhalten, das sich aus deren Differential- 


gleichungen mit Koeffizienten, die Funktionen des ungestorten Aus- 
gangswertes sind, ergibt. Das Obergangsverhalten solcher Systeme 
bei groBen Storungen kann fur den Entwurf von Bedeutung sein. Die 
Arbeit beschreibt die ersten Ergebnisse einer Untersuchung des tjber- 
gangsverhaltens solcher Regelsysteme und gibt deren Bezichungen zu 
Eigenschaften an, die sich als eine Erweiterung der Wurzelortskurven- 
Methode bei kleiner Storung betrachten laBt. 

Die Ergebnisse der proportionalen Regelung von Strecken, die eine 
Ubertragungsfunktion besitzen, die bei kleinen Storungen eine aus- 
gangsabhangige Verstarkung und entweder einen festen einfachen 
Pol Oder ein festes komplexes Polpaar haben, sind dargestellt. Hat die 
Regelstrecke einen einfachen Pol, so ist bei groBen Storungen die 
Sprungantwort ahnlich der eines linearen Systems erster Ordnung. 
Die Zeitkonstanten des fast aquivalenten linearen Systems werden 
daher als Polstellen der Wurzelcrtskurve bei kleinen Storungen abge- 
leitet. Fiir eine Regelstrecke mit komplexen Polpaaren kann die 
Sprungantwort bei groBen Storungen sowohl stabil als auch instabil 
sein. Die Stabilitatsbedingungen und die Form des stabilen Verlaufes 
werden nach verschiedenen Moglichkeiten untersucht, namlich erstens 
mit Hilfe der direkten Methode von Liapunov, zweitens durch die 
Darstellung in der Phasenebene, drittens nach einer Erweiterung der 
Wurzelortsmethode bei kleinen Storungen auf eine „Wurzelflache“ 
und schlieBlich mittels einer Naherungsmethode. Der Begriff der 
,,Wurzelflache“ scheint eine erfolgversprechende Untersuchungs- 
methode fiir derartige Regelsysteme zu sein; die Untersuchungen 
werden fortgefuhrt. 


Introduction 

A particular class of physical process is that whose input 0 and 
output 6 0 are related by a differential equation of the form 

£ (a m +a m0 d)e ( ^+^a 0 + a 00 e+ £ a Oj 0 u ^0 o 

k 

= b + b 0 e+ £ bjO U) (1) 

J = 1 

in which superscripts denote derivatives with respect to time, 
and any a or b may be negative or zero. As an example, the 
equation 

~dF + ^ ai ~ d) W~{ a ° od+ ^) e o =0 @) 

can represent the response of a nuclear reactor on a one-point, 
one delayed-neutron-group basis, where 0 o is the neutron 
population and 6 the reactivity. The differential equation of 
such a process is characterized by the presence of products both 
of 6 0 with derivatives of 6, and of derivatives of 6 0 with 9. 

The relationship between an input equilibrium level 0 e and 
the corresponding output equilibrium level 0 Oe is obtained from 
eqn (1) with all derivatives equated to zero: 

(a 0 + d 00 9 e ) 9 0e = b + b 0 6 e (3) 
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If 6 has a small variation <50 about 6 e , and 0 O has a resultant 
small variation <S0 O about 6 0e , linearization along with eqn (3) 
yields the following transfer function between ^(9 and 

k 

<500 G7) Z (b J- a 0Ae)P J 

- (4) 

I (a,n + a,n 0 de)p m 

m = 0 

the form of which is a function of O e and 6 0e . Use of eqn (3) to 
eliminate 6 e produces a small-signal transfer function which is 
a function of the unperturbed output level only. Any process 
whose dynamic response can be described by eqn (1) can thus, 
in a restricted sense, be described as an 4 output-dependent 
process’. 

For a feedback control loop round a process of this kind, 
such that 6 is given by (0$ — 0 o ) where 6 i is the input signal to the 
system, the shape of the roots-loci for small-signal response is 
a function of 0 Qe . A roots-surface’ can therefore be constructed 
in a three-dimensional space whose axes represent the real and 
the imaginary frequency components and 6 0e . From the shape 
of this surface, the small-signal response of the closed-loop 
system can be directly deduced for all values of 0 Oe and of 
loop gain. 

In this paper two special cases are investigated for the 
relations which exist between the characteristics of the roots- 
surface and the response of the closed-loop system for large 
variations in its state. On the basis of these relations design 
procedures for the control of processes of this class are being 
developed which are similar to those used in the roots-locus 
method of synthesis for linear systems and for systems which 
contain single non-linear elements. 

The Control of a Process Whose Transfer Function Consists of 
a Variable Gain and a Simple Fixed Pole 

Such a process is characterized by the relation 

dO 

-^+(a 0 + a 00 6) 9 0 = b 0 9 
which may be written as 


value of the output from the lag. In the subsequent consideration 
of this system a restriction 6 0 > — (1/A) is imposed, so that the 
process gain has only positive values. 

For any small perturbation ddj from the equilibrium state 
(Piet ^oe)? the transmission from 60$ to (3(9 0 obtained from 
eqns (3) and (4) is shown in Figure 2. 

Roots Surface 

To construct the roots-surface in the (<r, jco, X6 0e ) space 
(Figure 3\ the path of the open-loop pole - l/T(l + X6 0e ) is 
drawn first. This lies entirely in the (a, Xd 0e ) plane, so that the 
roots-surface in this case is simply the portion of the (<r, X6 0e ) 
plane on the negative side of the open-loop pole path. The root 



Figure 2. Block diagram for small variations in the process of Figure 1 



Figure 3. Roots-surface and root paths for first-order lag process 


T-^+(l-XK0)9 o =K0 

where b Q /a 0 = K, a 00 /b 0 = — A, and l/a 0 = T, and K, T, and X 
are all positive. Figure 1 is a form of block diagram for the 
closed-loop system around the process, which shows that it can 
be considered to consist of a first-order lag preceded by a gain 
element whose instantaneous value depends on the instantaneous 



Figure 1. Block diagram for process with variable gain and simple 
fixed pole 


paths lie in the surface at a distance K/T( 1 + X9 0e ) from the 
open-loop pole path; a few are drawn in Figure 3 for different 
values of K. The equation of a root path is 


i+K(i+xe 0e y 
T 1 + 29 


where a is the position of the root; each path has a maximum 
at X0 Oe = (i/iT'/a) — 1, for which 


2 

C7 “ T(l+W 0e ) 

and the locus of maxima is indicated on Figure 3. 


Large Step Response 

Consider a large step change in O t applied to the system at 
t = 0. The initial equilibrium values of and 0 Q for t < 0, 
denoted by 6 i0 and 0 oo respectively, are related by 

K9 i0 (l+18 oo )=0 oo (l+K(l+ A0 OO )) 
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and, for / > 0, 0$ = 6 if . The final equilibrium value for 6 0 , 
denoted by 0 O/ , is given by 

KB if (1 + W 0lf )=0 Of (l + K(l+ W 0f )) 

The system responds according to the equation 

d/5 

T-rp+(l +K—XKO tf ) e 0 + XKQl = K9 if 


This may be solved by separating the variables to give 


in which 




Of 


A- £e~ C t 
A + Ee~ Ct 


A = 1 + K (1 + W 0 /) (1 + X9 00 ) 

B = ( 1_ fe) (1+X(1+A0O/)) 


l + iC(l+10 o/ ) 2 
T(1 +X9 0f ) 


( 6 ) 


E=XK{9 0f -9 00 )( \ + X9 0f ) 

To correlate this response with the characteristics of the roots- 
surface, eqn (6) may be approximated by a single exponential 
relation 

0o = 0o/“(0o/"0oo) e t/Teq (7) 

where T eq is chosen to give in some sense an optimum representa¬ 
tion. If the chosen criterion is that the integral (0 O/ - 0 O ) d t 
should be the same for the exact solution (6) as for the approx¬ 
imate solution (7) (cf. Nechleba 1 ), the following result is obtained: 


= _i_ = _ *K(Ooo-e of ) 

t „ 

With reference to eqn (5), if the small-perturbation pole at 
9 0e = B 0 f is denoted by 07 , eqn (8) becomes 


*K(9 oo -0 of ) 

Tlog^U—t 0 *'- 000 ^ 


(9) 


This gives the result 




-CTy+—~(7 0 + 


_ ^(0Q/~0Q 0 ) 

3 r(i+20 o/ )(i+A0 oo ) 


( 10 ) 


where <r 0 denotes the small-perturbation pole at 6 0e = 6> 00 . 

The accuracy of representation by the equivalent time 
constants is illustrated in Figure 4 by the responses for K = 4, 
with X6 00 = ± 0-5 and X0 of — + 0-5 respectively; in each case 
the movement of the small-perturbation pole is considerable. 
It appears that the use of either equivalent pole is of adequate 
accuracy for engineering purposes in estimating the nature of 
large transients from the roots surface. 



Figure 4. Typical responses of the closed loop round the first-order lag 
process , with corresponding approximations from eqns (9) and (10) 


The Control of a Process whose Transfer Function Consists of a 
Variable Gain and a Complex Fixed Pole-pair 


This expresses the equivalent single pole as a function of the 
initial and final equilibrium values of 6 0 and of the small- 
perturbation pole position at 0 of . 

Furthermore, if | Ta f \ > | XK(Q Qf - <9 00 ) |, so that the 
expansion log (1 + x) = x - ix 2 -f }x 3 - ... converges, then 


This process is characterized by the relation 


d 2 0 o 

d t 2 


dfl, 


+ a i -£f+( a o + 000) 00 = b 0 9 


which may be written 


a 


eq 



where u — XK (0 of — 6 00 )/Ta f , which may be a preferable form. 
Morgan 2 has suggested an alternative criterion for T eq which, 
unlike the first, does not require the exact solution for 6 0 . It is 


eq 


Bo.f~Bno 



where 



_1_ 

00/-000 


* 9o/ dfl 0 

000 


■de 0 


"dpr + 2Cco„-~+co 2 (l-XK9)9 0 =co 2 K9 

where 

a i=2 £co n , a 0 =cal, a 00 =-XKco 2 , and b 0 =co 2 K, 

and K, co n , f, and X are all positive. Figure 5 is a form of block 
diagram for the closed-loop control system around the process, 
which shows that it can be considered to consist of a second 
order lag preceded by a gain element whose instantaneous value 
depends on the instantaneous value of the output from the lag. 
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Figure 5 . Block diagram for process with variable gain and complex 
fixed pole-pair 


S 0 i $0 


K (1 . X 9 0e )' 


1 + X6p e 


o cu ^ 

D + 2 i?U) n D + -^- 

* n 1 + X.6oe 


se r 


Figure 6. Block diagram for small variations in the process of Figure 5 


No restriction is imposed on 0 Q for this system, so that the 
process gain may at times be negative. 

For any small perturbation <5 0* from the equilibrium state 
(Pie j the transmission from 66 to 66 0 obtained from 

eqns (3) and (4) is shown in Figure 6. The small-perturbation 
natural frequency co e and damping £ e is defined by the relations 


co e 2 = co„ 2 /( 1 + Afloe) and C e =—-C 
Roots Surface 


To construct the roots-surface (. Figure 7), use is made of 
non-dimensional coordinates a/co n , jco/co n and X0 Qe . The 
positions F (= a -f jay) of the two open-loop poles are given by 

P = - C e CO e ± co e (C 2 -1)*== - Cco„ + ( 0 n (^ 2 - jq-F —■) 

For — 1 < X 0 Oe < (1/f 2 — 1), the real part — f co n of P is con¬ 
stant, and the imaginary part ± jco n (1/(1 + A6 0e ) — £ 2 )£ tends 
to infinity as X6 0e tends to — 1; and for X6 0e > (1/f 2 — 1) or 
X6 0e < — 1, the imaginary part ofFis zero. Figure 7 is drawn 
for f = 0-707, with the result that the roots-surface consists of: 


(a) for — 1 < X6 0e < 1, the portions of the cr/co^ = — £ 
plane ‘above and below’ the open-loop pole paths; 

(b) for X0 Oe > 1, the whole of the cr/co n = — f plane together 
with the portion of the jco/co n = 0 plane between the open-loop 
pole paths; and 

(c) for X0 Oe < — 1, the portions of the jco/co n — 0 plane 
‘to the left’ and 4 to the right’ of the open-loop pole paths. 

Figure 8 is complementary to Figure 7, presenting the parts 
of the roots-surface in the two normal planes separately. The 
positions of the closed-loop poles are 


-C(D n ±CD n [ C 


l+K(lFA6 0e ) : 


2 \* 


1 + X0 { 


Oe 


(ii) 


The root-paths, a few of which are drawn in Figures 7 and 8, 
lie in the real plane for X0 Oe < - 1; if K < £ 4 /4 the paths lie in 
the real plane for 


-!+—[C 2 -(C 4 - 4 lO 4 ] < A0 Oe < - 1 

+xh[c 2 +(c 4 -4K)-q 


but for K > £ 4 /4 the paths do not meet the real plane anywhere. 



Figure 7. Roots-surface and root paths for second-order lag process , 
with f = 0-707 



Figure 8. Two-plane presentation of roots-surface of Figure 7. (The 
negative half of the Gjco n = — 0*707 plane is the mirror image in the 
X0 0& axis of the positive half and is omitted) 

Large Step Response 

Consider again a large step change in 0$ applied to the 
system at t = 0 for an initial equilibrium state (0* o , 0 OO ) defined by 

K9 i0 (1 -F A0 OO ) = 0 O o (1 + K (1 + /W 00 )) 

The system responds to a disturbance = 6 if for t > 0 accord¬ 
ing to the equation 

+: 21 Cco "'W + '' (1 + K ~ 1K6 ‘^ 6 ° += v 

( 12 ) 
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This may be written more generally with <9 n = A0„ and 

6 if =Xe it : 


d 2 © 0 

~d?~ 


+ 2 


d©o 

dt 


+ co*(l + K-K& if )0 o + co*K&l = cotK& if (13) 

It is first necessary to investigate the stability of the system. 
For this Liapunov’s Direct Method is suitable, with eqn (13) 
rewritten in the form 


0 2 = co 2 K0 tf -2(co n 0 2 -co 2 n (l+K-K0 V ) 0, - co 2 K0 2 

where G 1 = G 0 and G 2 = d9 0 /dt. The system has two critical 
points for which Q 1 = 0 and 6> 2 = 0, namely,} 


(A) @ 1 =A 1 ,0 2 =O, and (B) 0 1 =A 2> @ 2 = 0 

where A x and A 2 are the roots (both real and distinct) of the 
quadratic 

K0 2 1 +(l+K-K0 if )0 1 -K0 if =O 
and A 1 > A 2 . 


Point A 

The origin is transferred to point A by the transformation 

0' 1 = 0 1 ~A U 0 2 = 0 2 (15) 

so that eqn (14) becomes 

^2=~co 2 (1 +K-K0 if +2KA 1 )0[ 

-2Cco n 0' 2 -co 2 K(0[) 2 

with the new origin now a critical point. The characteristic 
roots a x and a 2 of the linear approximation satisfy 

~~ a 1 
-co 2 (l+K-K0 if +2KA 1 ) -2£co„-a = ° 

from which + x 2 = - 2 f co n , which is negative, and 
«* =® n 2 (l + K- K0 if + 2 KAJ, which is positive. Thus 
both and x % have negative real parts, and point A is asymp- 
totically stable. 

Point B 


Stability and the Small-perturbation Roots-surf ace 

To each value of 6 if there correspond two possible equili¬ 
brium states (0 if , 6 0f ) given by the relation 

Kd if a+ m 0/ )=e 0 f (i+K(i+xe 0f )) ( 17 ) 

as shown in Figure 9. From the foregoing discussion of stability 
it follows that the higher value of 0 of is stable, and the lower is 
unstable. 

This result could also have been deduced from the roots 
surface of Figure 7, since for XQ 0e < — 1 there is a small-pertur¬ 
bation closed-loop pole in the ‘right-half space’, while for 
> — 1 the small-perturbation responses are stable. How¬ 
ever, the complete analysis given above provides the necessary 
justification for the use of the roots-surface in this way. 



Bounds of Stability 


Consider now what bounds exist on 6 if for a given initial 
equilibrium condition (either stable or unstable) to ensure that 
the response of 0 O is asympotically stable to the greater cor¬ 
responding value of 6 0f . The problem is to find a suitable 
Liapunov function for system (15). This by use of (17) may 
be written 


where 


<9' 1 = <9' 

0' 2 = - co 2 n R0[ -2 Cco n 0' 2 - co 2 K (0 J) 2 


The origin is transferred 10 point B by the transformation 

&1=0 2 -a 2 , 0’'=0 2 

Eqn (14) becomes 
0"=0" 

^2=~to 2 (l+K-K0 if + 2KA 2 )0^ 

- 2 Cco„0 2 - co 2 K (0’[) 2 

X- K — KQ if + 2 KA, 2 ) < 0, the characteristic roots 
ot the linear approximation are real and of opposite sign, so 
that point B is a saddle point. 


(16) 


R=l(K0 if +K-iy+4Ky 

In terms of fa = K0\/R and fa = K0'Jco n R’l, this becomes 
<j>i=(o n jR<}> 2 

<!>2 = - C0 n (^/Rfa +2 C<t> 2 + jR(j> 2 ) 


(18) 


with critical points at A: fa = (fa = 0, and at B: (j)-. = — 1, 
02 ~ indicated in Figure 10. 

Consider the function V 2 = 3 fa 2 + 3 fa + 2 <6 3 for 
which V 1 = —12 £co n fa (cf. Lasalle and Lefschetz 3 ); V lt being 
zero on the fa axis and negative elsewhere, is negative semi- 
, e n te ' ^J s symmetric about the (f> 1 axis, and, in the infinite 
half-plane for which fa > - 3/2, it is positive except at the 
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origin where it is zero. The closed region Q 1 formed by the 
indicated part of the V 1 = 1 contour is one of asymptotic 
stability to the origin, because within it (a) V 1 is positive (except 
at the origin where it is zero), (b) V x < 0, and (c) V t is not 
identically zero along any trajectory of the system (cf. Lasalle 
and Lefschetz 3 ). 



Figure 10. The region of asymptotic stability D 1 corresponding to V 1 

From the extent of Q x it is evident that the restrictions on 
the magnitude of the input step for stable response, for particular 
values of G i0 and K, are — 1 < K@' 10 /R < j. Corresponding 
to O Q0 = — 1, the critical value of <9' 10 is (<9 10 — Af) = 
(1 — — K0 if — R)/2 K. This critical value lies within this 

range, so two separate ranges must be considered: 

(a) (1 - K - KO if - R)/2 K < 0' 1Q < R/2 K and 

(b) - R/K < <9' 10 < (1 - K - KG if - R)/2 K 

These correspond to initial equilibrium states that are stable 
or unstable respectively. 

(a) Using (17), the condition for a stable response from a 
stable initial equilibrium state is 

K0 if +2 \_{K0 if + K- l ) 2 + 4 X] 4 > K0 iO 

+ [(K0 iO +K-l) 2 + 4Kf 

The corresponding allowable values of 0 if for stable response 
are shown in Figure 11. 

( b ) Using (17), the condition for stable response from an 
unstable initial equilibrium state is 

K0 if - [(K0 if + K- 1) 2 + 4 K.y< K0 iO 

-[( K0 iO + K-l) 2 +4Kf 



Figure 11. Bounds on @ if for stable response deduced from Q 1 and 0 2 
(Stability exists to the left of each boundary) 


This reduces to the simple inequality 0 if < @ i0 for all values 
of G i0 and positive K. Thus, in this case, the application of a 
negative input step results in a final stable equilibrium state. 

Less Restrictive Bounds 

The bounds on 0 if derived from V 1 are too restrictive, 
because they define ranges of allowable values for 0 if which 
are in some cases less than are actually permissible. Asa possible 
second Liapunov function we consider 

V 2 = A(j)\ + B(j) 2 + C^) 1 ^) 2 + ^0102 F F(p2 + 

in which the values of the coefficients A, B s ... are to be deter¬ 
mined. To produce a form for V 2 which is symmetric about both 
axes, one chooses 

i = 2(3iV + 8), J3 = 5iV + 8, C = ^N(N + 8), 

D=-y~N(N + 8 ), E= —4N, F=-2/3N 3/2 , 
and G = — 2 N, where N = 



Figure 12. Sketch of contours of V 2 and of V 2 = 0 for a value of 
N< 6 + 2 y 17 

This gives V 2 = £co„[(N 2 -12N-32)<pi 

+ N(N+8) (<f> 2 — 1) <f>%+2 N 2 (j>%<f>2] 

so that both V 2 and V 2 depend on the parameter N (whereas V 1 
has no parametric dependence). 

For values of N less than (6 + 2/17) == 14*25, there is an 
infinitely long region straddling the cj) 2 axis for which V 2 is 
negative, except at the origin where it is zero; this lies between 
the two branches of the V 2 = 0 curve which intersect the ^ 
axis perpendicularly at ± 1. The character of V 2 must be 
examined to discover if any contour forms a closed region 
within the region of negative V 2 . If this exists, a region of 
asymptotic stability with less restrictive bounds than Q ± may 
result. 

Figure 12 shows the character of V 2 for a value of N less 
than 8*0 approximately. Up to a limiting value of C = & 3 , 
contours V 2 — C form closed regions within the region of 
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negative V 2 , so that the region within V 2 = k% is one of asymp- 
to tic stability; however, this new bound represents little, if any, 
improvement over the original one. But because the saddle 
point 5 of the V 2 contours lies on the left-hand branch of 
a s indicated, it is possible, by combining Liapunov’s 
criteria with a deduction from eqn (18), to prove the existence 
of a larger region of stability. 

Consider the region formed by K a = /c 4 and closed by the 
portion CD of the curve V 2 = 0. Throughout this region V 2 
is negative, and no trajectory can leave the region across 
V 2 = ki. At any point on CD where = 0, a trajectory must 
run tangent to the V 2 contour through the point. Because CD 
lies in the region 4> 2 > 0, from eqn (18) fa > 0; hence, every 
trajectory from CD has a component of velocity in the positive 

direction. From inspection of the directions of the V 2 contours 
relative to CD, it is evident that all the trajectories cross CD 
into the region Q 2 , which is one of asymptotic stability. 

For N < 8-0 approximately, the contour V 2 = k 5 through B 
provides the limiting contour. The condition for a stable response 
from an initial stable equilibrium state is therefore 

K0 if +3 l(KQ lf +K-lf+AKf>K@ i0 

+ [(K0 iO +K-l) 2 +4Kf- 

The allowable values of 0 if arising from this expression, which 
are valid for N < 8-0 approximately (i.e. [K(O if -f 1) — l] 2 
< 4-0 (16 - A) approximately), are also shown in Figure 11. 

For 6 + 42^/17 > A > 8-0 approximately, the contour 
through B is unsuitable because it crosses the right-hand branch 
of the V 2 = 0 curve for a negative value of </; a . In this range of N, 
one of the intermediate contours between /<r 3 and k 5 which does 
not cross the branch in this way may be used to give improved 
bounds. 

The Phase Portrait 

As eqn (13) is a second order differential equation, its 
solutions can be portrayed in the phase plane formed by the 
coordinates X0 0 and a() 0 , to display their stability in a compact 
form. For a set of values of £, co n and K, there is a different 
portrait for each value of X6 if . However, if a linear transforma¬ 
tion on X6 0 is employed to place A x at (1, 0) and A 2 at (- 1, 0), 
the work involved in drawing portraits for different X6 if is much 
reduced. The transformation also gives the lower bound on X6 i0 
for stable response, but does not give the upper bound, unlike 
the Liapunov function analysis. 

To give a better appreciation of the nature of Q 1 and Q 2 in a 
particular case, the method of isoclines has been used to derive 
die phase portrait shown in Figure 13 for the process with 
C — 0-707, a>„ = 1, and K — l, and for X6 0f = 2. The two 
regions are superimposed; whilst fl 2 represents a large im¬ 
provement over Q lt it still underestimates the actual upper 
bound that exists on X d i0 . 


out the region of asymptotic stability. In both Q x and Q 2 , Kis 
only negative semi- definite, so no estimate of the time behaviour 
of the solutions is possible by this method. 

An attempt to obtain an analytical solution to eqn (13) in 
closed form through a transformation listed by Murphy 4 has 
been unsuccessful. An approximate solution has been obtained 
following Grensted 5 by assuming that 

0o = 0os+a sin \j/ 

where a(t) is the ‘amplitude’ and i j/ ( t ) the ‘phase’ of <9 0 . 



Figure 13. Phase portrait of controlled process with £ — 0-707, 
°)n — 1, K — 1, andXQ 0 f — 2 

We define 

li— — a/a or a = t 
and 

cd=\Jj or \j/=\ t 0 codt 

Then 

0o = 0of+Az sin (Jo co df + <£) 

where A and (j> are arbitrary constants. 

Substitution of this form for <9 0 in eqn (13) gives: 


Approximate Estimation of Transient Response 

It is known that V can give an estimate of the largest time- 
constant of the response, from the figure of merit 


[^ 2 0)2 T ~ 2 + On —~ sin (Jo ojdt + (p) 


n=(-v!V) min 


+ [ft) - 2 pco + 2 C©„co] cos (Jo codt+cj)) 


where 1/jj represents the largest time constant. However, a 1 2r ^ _ tU» 

finite value for rj is only achieved if Kis negative definite through- + - j ©„ A A e (1 - cos 2 (J 0 f co dt + tj>)) = 0 
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If the third term which decreases with time is ignored. 


and 


co 


2 l+i£(l + <9 0/ ) 2 . 2 • 

- 0 ),,- r ---- bIX -fi~-2Cco n jLi 


1 + 0, 


Of 


(A — £cO n + — (b/C0- 


: Coo n + 


d(co 2 )/dt 
4 co 2 


An iteration procedure for of and p follows, which converges: 
a>o = fi o = 0 

n 2 - l±KO±@of) 2 


C 0 i =co: 


1 + 0 0/ 

2.^/i+^(i + e 0/ ) 2 


co 2 =co; 


1 + 0 


Of 


-C 2 ) : H 2 = £co n 


®3 = ®2 : ^ = fi 2 


(19) 


Through neglect of the third term indicated above, the resulting 
expressions for co and p from eqn (19) are functions of K,G,co n 
and O of , but not also of t as initially assumed. 


Correlation with Roots-surface 

From a comparison of eqns (11) and (19), the equivalent 
approximate closed-loop poles for large transients are identical 
with the small-perturbation roots evaluated at the final value 
Mu- 

The accuracy of the approximation can be seen with reference 
to Figure 14. This shows three step responses of X0 o , obtained 
from an analogue computer, for the process with K=l, C=0*707, 



co n = 1. Eqn (19) gives that the equivalent damping ratio is con¬ 
stant, while the equivalent damped frequency depends on X 0 Qfm 
For the solutions of Figure 14, eqn (19) gives values for co/a) n of 
1*41, T68, and 1*93, while the actual values from the figure are 
almost constant at 1*59, 1*85, and 2*06 respectively: eqn (19) 
gives a constant value for ft of 0*707 compared with the computer 
values of 0*48, 0*56, and 0*62 (determined from the initial over¬ 
shoot), all of which tend to 0*707 as the oscillations are damped 
out. 


Conclusions 

The large step response of a controlled process whose small- 
perturbation transfer function consists of an output-dependent 
gain and a simple fixed pole is similar to that for a first-order 
linear system. The magnitude of the steady-state response de¬ 
pends in a non-linear way on the magnitude of the disturbance. 
For the transient response, time constants of nearly equivalent 
linear exponential form can be found in terms of the initial and 
final small-perturbation time constants; these describe the re¬ 
sponse with adequate accuracy for engineering purposes. 

The stability of response of a controlled process whose 
small-perturbation transfer function consists of an output- 
dependent gain and a complex fixed pole-pair depends on the 
magnitude and direction of the disturbance and on the initial 
equilibrium. By Liapunov’s Direct Method satisfactory general 
criteria have been found and in a particular case compared with 
a complete analysis by the phase-plane method. The ‘roots- 
surface’ associated with the system is incapable of predicting the 
bounds on the disturbance for stable response, which is an im¬ 
portant restriction on the ‘roots-surface’ method of analysis (cf. 
M’Pherson 6 ). 

For a disturbance within the bounds of stability the large 
step response is similar to that for a second-order linear system. 
The damping and frequency of nearly equivalent second-order 
response can be adequately expressed in terms of the features of 
the roots surface, as is verified in some particular cases by means 
of an analogue computer and by an approximate method of 
analysis involving time-varying amplitude and phase. 

The interest and support of the UKAEA (Winfrith ), and in 
particular of P. K. M'Pherson, Head of Dynamics Group , in con¬ 
nection with the project of which this work forms part is acknowl¬ 
edged with appreciation. 
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DISCUSSION 


Authors’ Opening Remarks 

Since our paper was submitted further Liapunov functions have been 
discovered which represent improvements on the functions V 1 and V 2 
described in the paper and illustrated by the regions Q ± and Q 2 in 
Figure 13. 

The restriction on the validity of V 2 to the range 0 < TV < 14-25 
limits its usefulness, whilst V 1 yields an over-conservative estimate of 
the region of stability. Using for greater convenience the following 
pair of equations in and <^ 4 , equivalent to eqns (18) 

h = CO n QR(j) A -2Z4> 3 ) 4> 4 =-CQ n 7jR0 3 (1 + (j) 3 ) 
where 

h=K0'jR and <; p 4 .=K(0' 2 lco ny /R + 20' 1 / y /N)lR 

the function V 3 has been found by the method of Zubov 1 and the 
functions V 4 and V 5 by the same method of undetermined coefficients 
as used for V 2 : 

r 3 =(<^ + <£l)/2£a>„ V 4 =cj)l + <j)l+24>ll3 
h= ~2(j)3(l+^/N(l)J2) V 4 = — 4 £co n 4>l (1 + (j> 3 ) 

Vs = <t>l + 4>l + [2 (N +2)-4 y/N+I] <561/3 N 

h = ~£&> n $3 [4+(4(AT +2)—8^jN + l)4> 3 IN 

+4(JN+1- 

In the selection of the forms for V sl and V 5 , the principal object has 
been to produce infinite half-planes of negative V instead of a bounded 
region as with V 2 ; at the same time, since explicit expressions for the 
extent of the derived region of stability are desirable, these forms have 
had to be of low order. The existence of a useful region of stability 
from V 4 has been proved by reasoning similar to that used for V 2 ; the 
existence of the region of stability from V 5 has been deduced from a 
theorem on Liapunov functions (Reference 3 of the paper. Theo¬ 
rem VI). 

Using as a convenient measure of the extent of the region the 
coordinate L n , where the Q n contour intersects the positive axis, 
the expressions for L l9 L Zi L 4 and Z 5 are 


Lt= 0-5; L 3 =2/y/N+4 -; 

L 4 = \yjh (W +12) (3 AT + 4) = (N +12)3/4 N 

L 5 = y3(N +8+4^JV+T)(3 AI + 8—4^/iV+T) 

- N - 8 - 4 VjVTT]/[4 (N +2) - 8 V^TT] 

As regards L 2 , it is shown in the section entitled ‘Less Restrictive 
Bounds’ that for TV < 8 the value of L 2 is 1-0, while its value for 
8 < TV < 14-25 is implied in the last paragraph of that section. 
Figure A shows the way in which these further functions remove the 
restriction on the value of TV required by V 2 , and successively extend 
the zone for which stability is assured. Figure B shows the zones 
*^3? ^4> *^s corresponding to these new functions in a particular 

case. 

Finally, it must now be said that the Introduction reveals an 
underestimation of the difficulty of achieving general design techniques 
for a non-linear system, and it would be more appropriate to replace 
the phrase ‘design procedures ... are being developed’(in the last 
paragraph), by ‘design procedures... are being sought’. However, it is 
believed that this study throws’ valuable light on the behaviour of two 


output dependent processes, and as a result good progress is being 
made in the study of other processes of the class. 

Reference 

Zubov, V. I. Mathematical methods of investigating automatic 
regulating systems (Leningrad, 1959), USAEC transl. AEC-tr-4494 
(September 1961) 64-68 



Figure A. L n as a function of TV, derived from V n > showing a measure 
of the extent of the regions of stability derived from different Liapunov 
functions 



Figure B. Phase portrait of controlled process with £ = 0-707, co n = h 
X = l, A6 0 f = 10, and TV =22*16, with regions of stability from 
V 3f K 4 , and V 5 

P. K. M’Pherson, C. & I. Division , U.K.A.E.A ., Winfrith, Dorset , 
England 

In commenting on this paper, I would like to relate this and similar 
work to the needs of the control scientist who is working in the field 
of process control. My own field is in nuclear reactor technology and 
the examples I take must come from there. 
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This session is called ‘General Problems’ and, with little exagger¬ 
ation, the most general problem as seen in the field sometimes seems 
to be to find a part of control theory that is both accessible and ap- 
plicable. For example: 

(1) As an exercise an attempt was made to design an optimum bang- 
bang controller for a large nuclear power reactor. I allowed no 
approximation or simplification to be made in the representation of 
the plant it was to be a real engineering exercise to see if phase plane 
analysis would work. It did not. Not only was the problem intractable 
analytically (we could not reduce the problem to less than 12 dimen¬ 
sions) but we doubted if any existing digital computer could handle 
adequately a numerical treatment of the analysis. 

(2) The present paper before us, where we have heard that the 
‘roots-surface’, or a piecewise-linear approach, has failed to provide 
useful data on stability in the large for anything but the simplest cases. 

Some people might say at this point that the problem has been 
solved using Liapunov techniques. I agree that Liapunov functions 
have been found for what the plan dynamicist would call trivial cases. 
In general these exercises start with the following equations to re¬ 
present a nuclear reactor: 

n =(^)n + Zl iCi + S 

C i i ^-i c i 

k =k-YrjTj 
f J = a 1 n — g j T J 

n neutron density 

d concentration of ?th group of delayed neutron emitters 
S neutron source 

ft yield of delayed neutrons of zth group 

P -Sft 

k decay constant of it h group of delayed neutron emitters 
k reactivity 

/ neutron generation time 
Tj temperature of/th region in the core 
r p a p 8j constants 

Liapunov functions are known for the impulsive input case 
k ( t ) = hd (0 + £ rjdjQ 91 * 


which show stability in the large for the show-down and all-power 
cases. For example that due to Lipkin, and Ergin 


Vi=»+I< 


2 la i 


K<0 


if 


rjaj>0 

h >0 
kj< 0 


Letov’s and Lurie’s methods may be used to find other Liapunov 
functions. Smets has studied nuclear reactor stability using topological 
methods, describing functions and straightforward analysis, as well 
as Liapunov. He has also considered some special types of non-linear 
feedback. But for the more complicated cases delayed neutrons are 
usually omitted which is not of much use to the control engineer who 
relies on these delayed neutrons to make the reactor controllable. 

What I wish to show is that, admirable though this work is, the 
systems studied cannot be imagined to represent a real nuclear reactor 
plant with its many interactions and non-linearities. Figure A shows a 
block diagram of a real nuclear reactor, being representative of the 
dynamics of a boiling water reactor. 

It is full of non-linearities (particularly the boiling channel), trans¬ 
fer delays and feedback which dominate the dynamic behaviour 
characterized by the previous equations which, in fact, only represent 
the top loop round neutron production. 

One would like to have a method of analysis which would indicate 
comparatively quickly the stability in the large of these multivariable, 
multiloop, non-linear systems without having to spend considerable 
effort on analogue or numerical analysis. The only theory that is 
accessible to this class of problem at the moment is the much despised 
linear theory. So it seems reasonable to ask Professor Maclellan and 
Mr. Kerr to devise a method which would give us information on 
stability in the large from a consideration of the type of linear models 
used in system synthesis. Although they have given us some very 
interesting results, and all of us concerned with the project have 
acquired a considerably deeper insight into non-linear stability prob¬ 
lems, one must conclude that this approach is unlikely to yield the 
results that had been hoped for. So the field is still wide open, and one 
begins to wonder which approach might be better: a full-scale attack 
on Liapunov to find the rules with which to provide Liapunov functions 
for this class of multiplicative output dependent system, or resort to 
numerical analysis any time an analytical difficulty is encountered. 

I would like to end by asking the following intentionally provocative 
question: in view of the fact that numerical analysis on digital com¬ 
puters can reduce the most complex engineering problems to a tractable 
form, is it worth devoting any significant research effort to the devel¬ 
opment of analytical methods of solution which at the best are only 
likely to provide qualitative answers? 



Figure A. Simplified flow chart for boiling water reactor plant 
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C. N. Kerr and G. D. S. Maclellan, in reply 


We noted with interest that T. J. Williams also drew attention to this 
general problem in his Congress Survey Paper, in the sections entitled 
‘The Uses of Process Dynamics in Process Plant Studies’ and ‘The 
Formulation and Use of Theoretical Models’. We agree with Mr. 
M’Pherson that the locally linearized approach to analysing systems 
of the output-dependent class has so far proved to be of little value 
except in the simplest cases. Xt is appropriate here to point out that 
the small-signal, output-dependent transfer function of any process 
may not be unique to that process, and in consequence the method 
inevitably has its limitations. For example, the following transfer 
function 

Mo (p) n P + a oo 
<50O) 0e p(p + a 1 ) 

and the corresponding roots-surface belong to both the process in 
eqn (2) of the paper and the process described by the equation 


d% 

d t 2 


+ Cl\ 


dOo 
d t 



0 0 =0 


The large-scale transient responses of these different controlled pro¬ 
cesses will obviously differ, yet any deductions from the features of the 
roots-surface would apply to each. 

Since a general locally linearized approach to these systems 
appears to be unfruitful, Mr. M’Pherson poses his challenging question. 
In our opinion, it is still desirable to employ whatever analytical 
methods are valid and useful to achieve general results in analytical 
form, even though such methods may be more complex than the 
existing linear methods. We feel that the effort expended in utilization 
of Liapunov’s method, for example, pays off finally by providing 
definite quantitative information about the behaviour of systems of 
this class, as is demonstrated by the case of the second-order controlled 
process of the paper. 


C. N. Shen, Mechanical Engineering Department , Rensselaer Poly¬ 
technic Institute , Troy, New York , U.S.A. 

The authors of this paper have presented lucid examples of control 
theory techniques. An output equilibrium level d 0e can be expressed 
in terms of an input equilibrium level 6 e and vice versa. It seems to 


me that in eqn (4) of the paper the use of eqn (3) to eliminate 6 0e 
instead of 6 e becomes an ‘input-dependent process’. Would the authors 
care to discuss the word ‘output’ in the title of their paper? 

There is no equilibrium level 0 Oe for the reactor kinetics in eqn (2). 
The neutron density for a critical reactor could take any level at 
equilibrium condition, thus eqn (3) should be modified. The application 
of the authors’ method to the problem of reactor kinetics with six 
groups of delayed neutrons would be of great interest. In addition, it 
would be helpful if the authors would indicate the physical significance 
of their integral criterion on the accuracy of the result with reference 
to Figure 4. 

I believe that the problems for output dependent processes are 
very important. Thus the purpose of the discussion is to generate 
interest in this field of study. 

C. N. Kerr and G. D. S. Maclellan, in reply 

Professor Shen is correct in stating that eqn (3) may be used to 
eliminate 0 Oe rather than 6 e from eqn (4). However, a situation may 
arise, as he points out, where the value of 6 e is zero; this occurs if the 
transfer function has a pole at the origin. In such an equilibrium state, 
the output may have any value, and the transfer function depends 
only on 6 0e . Furthermore, for the two processes considered the block 
diagrams of Figures 1 and 5 show how the instantaneous gain can be 
regarded as directly dependent on the output. For these reasons, the 
general class of process is regarded as output, rather than input, 
dependent. 

There is no need to modify eqn (3) for the case of a nuclear reactor 
in which the coefficients a Q , b and b o are zero. At equilibrium, either 6 e 
or 6 0e (or possibly both) must be zero, and the only non-trivial situation 
is that where the equilibrium neutron flux is non-zero. 

We agree that an investigation of the reactor with all six groups of 
delayed neutrons would be of considerable interest. However, it has 
been found helpful to start this general investigation with processes of 
lower order, and it is intended to extend the investigation successively 
to higher orders. 

The time integral of the transient deviations of the output from its 
final value provides a convenient measure of the quality of the response. 
By equating the value of this integral arising from the approximate 
solution to the value from the exact solution, a ‘best’ approximation 
is obtained. 



692 



REPORTS 


B. N. PETROV* 


Automatic control theory is a new field which in recent years 
has been making exceptionally rapid progress and has undergone 
radical qualitative changes. 

We have witnessed fundamental developments in the theory 
of non-linear system stability and in the theory of linear and 
non-linear sample-data systems. New design approaches have 
been suggested for self-adjusting and self-adaptive systems, 
constituting a new type of automatic control. The interest of the 
specialists has been focused on optimal control. Various tech¬ 
niques for the design of optimal systems based on different 
performance criteria have been developed, notably, Pontryagin’s 
maximum principles and Bellman’s dynamic programming. 

Stochastic methods have found their way into practically 
every field of automatic control theory. More attention has been 
devoted to structural problems and to hybrid and invariant 
systems; the sensitivity problem has been formulated; new 
results have been obtained in finite automata and switching 
circuits theory. 

Principles of self-organization and learning machines, 
problems of pattern recognition and the ‘man-machine’ problems 
have lately been extensively investigated. These as well as 
bionics and the theory of large-scale systems are purely cyber¬ 
netic matters, which broadened the scope of modern automatic 
control. 

Remarkable survey papers on fundamental problems of 
automatic control which cite most important recent results in 
our field of art, have been a very valuable contribution to the 
scientific programme of the Congress. 

The Second I.F.A.C. Congress has clearly demonstrated 
these qualitative changes and marked progress in automatic 
control theory. Moreover, it is equally important to stress the 
fact that this meeting has provided an opportunity for the 
scientists from all parts of the world to discuss in friendly 
atmosphere the problems they are working on. 

In the brief summary, which I have the privilege to deliver 
to the Congress, it is not possible to refer to the authors person¬ 
ally and to evaluate new results which have been reported. 
Therefore I offer my apologies to the speakers and will con¬ 
centrate only on main issues, being unable to cover all branches 
of automatic control theory presented here. 

The Congress was high-lighted by the concepts of optimal 
control theory. Optimal control has made essential progress 
and optimal control technique has been generalized for many 

* Chairman of the I.F.A.C. Technical Committee on Theory. 


types of automatic control systems (such as, sample-data, 
distributed parameters, time-lag and digital systems, not to 
mention self-adaptive control where optimal and sub-optimal 
processes are inherent). 

Control algorithms based on various performance criteria for 
different plants, including industrial processes and utility systems, 
have been cited in many papers and digital computers are widely 
used for practical implementation of these algorithms. 

The stability problems have been solved for a number of new 
types of control systems, and particularly for self-adaptive 
systems. Liapunov’s methods, created more than 70 years ago, 
had a spectacular revival with the advent of these new types of 
systems and with the progress in computational procedures and 
equipment. It has been shown that optimal controllers designed 
on this basis and utilizing dynamic programming or maximum 
principle are inherently stable. 

These results are obtained in a number of papers devoted to 
the control system synthesis and analytical design. 

Although the Congress has not been marked by essentially 
new methods of statistical control system theory, the papers 
and their discussions have demonstrated everspreading appli¬ 
cations of stochastic approaches in the control system analyses 
and synthesis, and in the evaluation of optimal control laws. A 
new theory of dual control has been developed which makes use 
of decision function theory. 

It is extremely interesting to note that many papers reflect 
further research of the problems reported at the First I.F.A.C. 
Congress and solve a number of problems which were the topic 
of dicussions in Moscow. 

Sample-data and digital systems have received great atten¬ 
tion at this Congress. Papers dealing with foundations of non¬ 
linear sample-data theory show the way to investigate stability 
of the systems to evaluate their performance and periodical 
oscillations when different types of non-linearities are present. 

More effort has been directed to the solution of structural 
problems and to the search for new principles of automatic 
control, combining high accuracy with low sensitivity to para¬ 
meter variations and invariancy to external disturbances. 

Here we have witnessed new results associated with optimal 
control problems. New important features and efficiency of 
variable structure systems have been investigated, and design 
technique for invariant sample-data and hybrid systems devel¬ 
oped. A new problem of configuration automatic control for com¬ 
plex systems has been stated and a way to its solution proposed. 
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The well-established theory of linear systems has made new 
progress. I mean the solution of the quadratic, estimates inversion 
problem. 

Papers devoted to self-adaptive systems reflect essential 
progress in the state of art. Professor Truxal, who expressed 
his disappointment in the absence of new results in this field 
during the past six years, has every reason to be satisfied. I am 
referring to the papers in which Liapunov’s methods, describing 
function technique and other approaches, were used to evaluate 
self-adaptive system stability, the way to speed up an optimum 
mode of operation was proposed, the fundamentals of indentifi- 
cation theory were investigated and the number of other effective 
techniques were developed. 

Even more papers deal with analysis and design for particular 
types of self-adaptive systems. 

Many participants at the Congress were involved in the 
discussion of learning machines and systems, this being practi¬ 
cally the first opportunity to consider these problems at large. 


A number of interesting papers dealt with new cybernetic lines 
of thought in automatic control theory as well as to the ‘man- 
machine’ problem. These papers contain the basic theory with 
a number of specific block-diagrams and examples of learning 
machines which will find a wide application in future. 

It is to be regretted that at the Congress problems of bionics 
have not received the attention they deserved. 

In spite of the fact that the symposium on relay systems and 
finite automata was held less than a year ago, the papers in this 
field presented some new concepts. These papers considered 
once more that the theory of finite automata is being incorporated 
in the general automata control theory. 

As a large river taking numerous tributaries brings its 
waters to the sea, so the general theory of automatic control, 
forming from the early independently-developed branches of 
technical sciences and enriching constantly with new methods, 
gives all its power and possibilities to the technical progress to 
the better future of mankind. 


J. H. WESTCOTT* 


It seems to me that we are at present in a period of consolidation 
of theory, and this does not lead to quite such a spectacular show 
of sparks as in the period before it, when new ideas are flashing 
around everywhere. About 5 or 6 years ago we reached the 
period in which the river of our endeavour in theory had become 
very thoroughly frozen, although we were not particularly aware 
of it. Perhaps we were rather satisfied with the easy way in which 
we could skate over the difficulties that lay below, but neverthe¬ 
less, underneath subtle movements were at work and these 
movements began to generate a growing amount of heat as 
discussion broadened, until about the time of the founding of 
IFAC, the whole river began to break up into icefloes. I regret 
to say that some of our unfortunate colleagues were uncere¬ 
moniously ducked into the water and some, it is rumoured, 
were carried out to sea never to be heard of again. This was a 
time of great fluidity; old theories were brought out again and 
dusted; new theories were being launched every month; the 
chase was on and we were sometimes a little intoxicated and a 
little incautious. By the time of the first Congress in Moscow 
the river was flowing in a new direction. From all countries there 
were scientists and engineers who often independently were 
working with the new ideas of extremal control, whether in the 
form of actual equipment, simulated systems or in the currency 
of the mathematical ideas themselves. Each group had its own 
terminology, its own preferences, its own method of framing the 
problem. In the river there was still one or two icebergs floating 
about and nobody knew for sure whether they were ‘band- 
waggons-for-getting-on’, iconoclasts of destruction, or just a 
bad dream. They have now done their work of sweeping away 
some of the excrescenses and have for the most part sailed out 
to sea leaving a certain amount of untidiness behind them. 

In the three years since then we have seen a good deal more 
order return to the river; some traffic is beginning to flow towards 
the market place. There are still many pockets of undeveloped 
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territory, there are even one or two misconstructions, which may 
have to be dismantled; but by and large the immediate impasse 
has been breached and we can none of us complain of having 
nothing to do. Let me take you on a brief tour of the river as I 
now see it, and since we are in Basle we shall make our crossing 
by one of those ingenious and picturesque ferry boats which 
have no engine or source of power but rely entirely on the skill 
of the steerman to exploit the prevailing currents. 

The main current of progress of the Congress, in theory, 
appears to have two branches: 

(a) uses of the variational principle; 

(b) uses of statistical estimation theory. 

The first gives us optimal theory; in the noise-free case using 
trajectory methods. The methods of characteristics enables us 
to split the equations into a set of dynamic state equations and 
an adjoint state of co-state equations. Thus we have a two-point 
boundary value problem. 

The real difficulty lies in the numerical solution of problems 
of realistic size and complexity. There is much stumbling here 
and I sense that little shows for the prodigious effort that is, 
and needs to be, made to establish method. It is possible that 
a full-scale attack by specialist numerical analysts would save 
much unrequited effort here, although there are reasons to 
doubt whether this would be successful without the incentive 
of engineering motivation. So perhaps we shall have to struggle 
on our own. The real target here is not just a numerical solution, 
but a genuine synthesis method, preferably realized in feedback 
form. I think by the next Congress we will have such a method. 

The second branch deals with control in the presence of ran¬ 
dom disturbances; here the remedy of the method of character¬ 
istics is denied us and the minimizing partial differential equation 
must be solved directly. The Chapman-Kolmogorov equation 
shows how the random effects are diffused through the system 
modified by its dynamic characteristics. Our numerical difficul¬ 
ties, even under the simplification implied by the Fokker- 
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Planck equation, are now increased. The handling of the 
stochastic case (that is under conditions of statistical equili¬ 
brium), is near to the practical limit to achievement here. 

Finally, to take the problem of greatest practical interest, 
and hence, it always seems, maximum mathematical difficulty, 
we come to the fully adaptive problem. Here we have to intro¬ 
duce the idea of statistical estimation of possibly-varying 
parameters. The identification problem is one aspect of this 
and we have had some very practical contributions on this 
subject. In making control manoeuvres in an adaptive system we 
have a double motive: 

(a) to improve estimation of parameter values; 

( b ) to improve control using current parameter values. 

The compromise has to be made using statistical decision 
theory, and we have received helpful suggestions for this. I would 
like to make a personal comment on theoretical developments. 
There are three possible ways of exploiting mathematical theory: 

(1) to make use of existing theory; 

(2) to re-invent old theory; 

(3) to invent new theory. 

I put them in order of frequency of occurrence. Now in using 
variational methods we are clearly using established theory, 
but as we have seen when wishing to become numerate, we have 
had difficulties. In struggling against these difficulties we have 
often re-invented old theory. This is no reflection on the quality 
of this work, for it was no more easy to develop for a second 
time; but we have been in this numerical difficulty before and 


much valuable work lies unremembered in the old works on 
the numerical solution of differential equations. It could well be 
worth while to go through the vaults before embarking on the 
prodigious labour of outright invention. 

I have talked about the papers presented to the Congress. 
There is another facet of the Congress which is perhaps more 
important to workers in theory than in other spheres since they 
deal in a currency of ideas and that is the question of making 
personal contacts with other theorists. I have found this aspect 
of the Congress most valuable, but I cannot disguise from myself 
that it is still a difficult and time-consuming activity finding the 
man one wants to talk to. Between sessions a sort of Brownian 
motion sets in. As a distinguished colleague said, using what 
Professor Gerecke called the international language of 1FAC, 
‘all people are chasing each other trying to meet. It means, 
although I should not say it, half the time you are talking to 
the wrong people’. This is still one of the unsolved problems 
of congresses. 

Professor Truxal has let you into the secret of these light 
signals on the speakers’ rostrum. Fortunately for me they have 
not had time to fit the new one that he suggested should be 
added. During the first session of the Congress, when I presented 
my paper, the secret of how these lights worked was such a well 
kept one that even the chairman did not know it. In his des¬ 
peration at my loquacity, he crept up behind me without my 
noticing, as I was in full spate, and as I paused for breath said 
‘Quit stalling, and give us the new stuff’. I can think of no better 
slogan for the next Congress to speed the parting guests at the 
end of this Congress, than ‘quit stalling and give us the new 
stuff’. 
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